Tensor Factorization Techniques Research Articles

Direct Numerical Solutions to Stochastic Differential Equations with Multiplicative Noise.

Inspired by path integral solutions to the quantum relaxation problem, we develop a numerical method to solve classical stochastic differential equations with multiplicative noise that avoids averaging over trajectories. To test the method, we simulate the dynamics of a classical oscillator multiplicatively coupled to non-Markovian noise. When accelerated using tensor factorization techniques, it accurately estimates the transition into the bifurcation regime of the oscillator and outperforms trajectory-averaging simulations with a computational cost that is orders of magnitude lower.

Neighbor Graph Based Tensor Recovery For Accurate Internet Anomaly Detection

Detecting anomalous traffic is a crucial task for network management. Although many anomaly detection algorithms have been proposed recently, constrained by their matrix-based traffic data model, existing algorithms often suffer from low detection accuracy. To fully utilize the multi-dimensional information hidden in the traffic data, this paper uses the tensor model for more accurate Internet anomaly detection. Only considering the low-rank linearity features hidden in the data, current tensor factorization techniques would result in low anomaly detection accuracy. We propose a novel Graph-based Tensor Recovery model (Graph-TR) to well explore both low-rank linearity features as well as the non-linear proximity information hidden in the traffic data for better anomaly detection. We encode the non-linear proximity information of the traffic data by constructing nearest neighbor graphs and incorporate this information into the tensor factorization using the graph Laplacian. Moreover, to facilitate the quick building of neighbor graph, we propose a nearest neighbor searching algorithm with the simple locality-sensitive hashing (LSH). Besides only detecting random anomalies, our algorithm can also effectively detect structured anomalies that appear as bursts. We have conducted extensive experiments using Internet traffic trace data Abilene and GÈANT. Compared with the state of art algorithms on matrix-based anomaly detection and tensor recovery approach, our Graph-TR can achieve higher Accuracy and Recall.

Least-square approach for singular value decompositions of scattering problems

It was recently observed that chiral two-body interactions can be efficiently represented using matrix factorization techniques such as the singular value decomposition. However, the exploitation of these low-rank structures in a few- or many-body framework is nontrivial and requires reformulations that explicitly utilize the decomposition format. In this work, we present a general least-square approach that is applicable to different few- and many-body frameworks and allows for an efficient reduction to a low number of singular values in the least-square iteration. We verify the feasibility of the least-square approach by solving the Lippmann-Schwinger equation in a factorized form. The resulting low-rank approximations of the $T$ matrix are found to fully capture scattering observables. Potential applications of the least-square approach to other frameworks with the goal of employing tensor factorization techniques are discussed.

Bayesian Semiparametric Covariate Informed Multivariate Density Deconvolution

Estimating the marginal and joint densities of the long-term average intakes of different dietary components is an important problem in nutritional epidemiology. Since these variables cannot be directly measured, data are usually collected in the form of 24-hour recalls of the intakes. The problem of estimating the density of the latent long-term average intakes from their observed but error contaminated recalls then becomes a problem of multivariate deconvolution of densities. The underlying densities could potentially vary with the subjects’ demographic characteristics such as sex, ethnicity, age, etc. The problem of density deconvolution in the presence of associated precisely measured covariates has, however, never been considered before, not even in the univariate setting. We present a flexible Bayesian semiparametric approach to covariate informed multivariate deconvolution. Building on recent advances on copula deconvolution and conditional tensor factorization techniques, our proposed method not only allows the joint and the marginal densities to vary flexibly with the associated predictors but also allows automatic selection of the most influential predictors. Importantly, the method also allows the density of interest and the density of the measurement errors to vary with potentially different sets of predictors. We design Markov chain Monte Carlo algorithms that enable efficient posterior inference, appropriately accommodating uncertainty in all aspects of our analysis. The empirical efficacy of the proposed method is illustrated through simulation experiments. Its practical utility is demonstrated in the afore-described nutritional epidemiology applications in estimating covariate adjusted long term intakes of different dietary components. An important by-product of the approach is a solution to covariate informed ordinary multivariate density estimation. Supplementary materials include substantive additional details and R codes are also available online.

ProTICS reveals prognostic impact of tumor infiltrating immune cells in different molecular subtypes.

Different subtypes of the same cancer often show distinct genomic signatures and require targeted treatments. The differences at the cellular and molecular levels of tumor microenvironment in different cancer subtypes have significant effects on tumor pathogenesis and prognostic outcomes. Although there have been significant researches on the prognostic association of tumor infiltrating lymphocytes in selected histological subtypes, few investigations have systemically reported the prognostic impacts of immune cells in molecular subtypes, as quantified by machine learning approaches on multi-omics datasets. This paper describes a new computational framework, ProTICS, to quantify the differences in the proportion of immune cells in tumor microenvironment and estimate their prognostic effects in different subtypes. First, we stratified patients into molecular subtypes based on gene expression and methylation profiles by applying nonnegative tensor factorization technique. Then we quantified the proportion of cell types in each specimen using an mRNA-based deconvolution method. For tumors in each subtype, we estimated the prognostic effects of immune cell types by applying Cox proportional hazard regression. At the molecular level, we also predicted the prognosis of signature genes for each subtype. Finally, we benchmarked the performance of ProTICS on three TCGA datasets and another independent METABRIC dataset. ProTICS successfully stratified tumors into different molecular subtypes manifested by distinct overall survival. Furthermore, the different immune cell types showed distinct prognostic patterns with respect to molecular subtypes. This study provides new insights into the prognostic association between immune cells and molecular subtypes, showing the utility of immune cells as potential prognostic markers. Availability: R code is available at https://github.com/liu-shuhui/ProTICS.

Group travel behavior in metro system and its relationship with house price

Modeling travel behavior inter-relationship is important to traffic planning and management as well as spread modeling of infectious disease and information. Our ability to understand inter-traveler behavior has remained unsatisfactorily limited due to the lack of empirical travel-together data. Taking advantage of access to a large-scale smart-card dataset from the Beijing metro network, we investigate a KS test-based method to identify group travel and study the spatial–temporal distribution of group travel as well as their interaction with socio-economic attributes. Origin and destination stations, departure time, and travel dates are separately classified into several types according to the passenger flow adopting a tensor factorization technique. The results suggest that temporally passengers tend to travel in groups in the afternoon and on the weekend; spatially areas with moderate house prices are the least source for group demands, while areas with a relatively low house price are the most favorite place for group trips. This novel discovery can help to understand the impact of socio-economic attributes on intra-urban group/passenger flow movement.

Parallel tensor factorization for relational learning

Link prediction is a statistical relational learning problem that has a variety of applications in recommender systems, expert systems, and knowledge bases. Numerous approaches have already been devised to solve the problem. Tensor factorization is one of the ways to solve the link prediction problem. Many tensor factorization techniques have been devised in the last few decades, including Tucker, CANDECOMP/PARAFAC, and DEDICOM. RESCAL is one of the famous tensor factorization technique that can solve large scale problems with relatively less time and space complexity. The time complexity of RESCAL can further be reduced by making it parallel. This variant can also be applied to large scale datasets. This article focuses on devising a parallel version for RESCAL. A decent decrease in execution time has been observed in the execution of parallel RESCAL.

Graph regularized Bayesian tensor factorization based on Kronecker-Decomposable dictionary

The tensor factorization techniques preserve the intrinsic structure information better than the matrix factorization technique. However, the outliers on each mode are usually non-independent and identically distributed (non-i.i.d) and the incoherence is non-uniform. The existing matrix factorization methods cannot deal with the non-i.i.d outliers and the non-uniform incoherence well. Although Kronecker-Decomposable sparse dictionary learning (KDSDL) model has shown effectiveness in dealing with the non-uniform incoherence, the non-i.i.d outliers and the correlation between different mode are still not considered. To address these problems, we propose a graph regularized Bayesian tensor factorization based on KDSDL model. First, to consider the non-i.i.d noise in real scenario, we embed the mixture of outliers in each mode through KDSDL model; Second, to capture the correlation between different modes, a novel graph sparsity-inducing regularization is proposed. The experiments on tensor data with complex outliers demonstrate the superiority of the proposed methods over the existing state-of-the-art methods.

Modeling user preference dynamics with coupled tensor factorization for social media recommendation

An essential problem in real-world recommender systems is that user preferences are not static and users are likely to change their preferences over time. Recent studies have shown that the modelling and capturing the dynamics of user preferences lead to significant improvements on recommendation accuracy and, consequently, user satisfaction. In this paper, we develop a framework to capture user preference dynamics in a personalized manner based on the fact that changes in user preferences can vary individually. We also consider the plausible assumption that older user activities should have less influence on a user’s current preferences. We introduce an individual time decay factor for each user according to the rate of his preference dynamics to weigh the past user preferences and decrease their importance gradually. We exploit users’ demographics as well as the extracted similarities among users over time, aiming to enhance the prior knowledge about user preference dynamics, in addition to the past weighted user preferences in a developed coupled tensor factorization technique to provide top-K recommendations. The experimental results on the two real social media datasets—Last.fm and Movielens—indicate that our proposed model is better and more robust than other competitive methods in terms of recommendation accuracy and is more capable of coping with problems such as cold-start and data sparsity.

Unveil stock correlation via a new tensor-based decomposition method

Portfolio allocation and risk management make use of correlation matrices and heavily rely on the choice of a proper correlation matrix to be used. In this regard, one important question is related to the choice of the proper sample period to be used to estimate a stable correlation matrix. This paper addresses this question and proposes a new methodology to estimate the correlation matrix which does not depend on the chosen sample period. This new methodology is based on tensor factorization techniques. In particular, combining and normalizing factor components, we build a correlation matrix which shows emerging structural dependency properties not affected by the sample period. To retrieve the factor components, we propose a new tensor decomposition (which we name Slice-Diagonal Tensor (SDT) factorization) and compare it to the two most used tensor decompositions, the Tucker and the PARAFAC. We have that the new factorization is more parsimonious than the Tucker decomposition and more flexible than the PARAFAC. We apply our methodology to simulated datasets using different simulation parameters. Results are robust to different simulation settings and confirm the stability of the correlation matrix generated for two independent samples. The proposed tool applied to two independent samples of empirical data shows that the correlation matrices generated have a block structure representing stock industries. Furthermore, in accordance to two non-parametric tests, namely Kruskal–Wallis and Kolmogorov–Smirnov tests, the correlation matrices are statistically time invariant and hence, stable. Since the resulting correlation matrix is characterized by stability and emerging structural dependency properties, it can be used as alternative to other correlation matrices type of measures, including the Pearson correlation.

Robust CP Tensor Factorization With Skew Noise

The low-rank tensor factorization (LRTF) technique has received increasing popularity in data science, especially in computer vision applications. Many robust LRTF models have been presented recently. However, none of them take the skewness of data into account. This letter proposes a novel LRTF model for skew data analysis by modeling noise as a Mixture of Asymmetric Laplacians (MoAL). The numerical experiments show that the new model MoAL-LRTF outperforms several state-of-the-art counterparts. The codes for all the experiments are available at https://xsxjtu.github.io/Projects/MoAL/main.html .

Weighted aspect-based opinion mining using deep learning for recommender system

The main goal of Aspect-Based Opinion Mining is to extract product's aspects and the associated user opinions from the user text review. Although this serves as vital source information for enhancing rating prediction performance, few studies have attempted to fully utilize it for better accuracy of recommendation systems. Most of these studies typically assign equal weights to all aspects in the opinion mining process, however, in practices; users tend to give different priority on different aspects of the product when reaching overall ratings. In addition, most of the existing methods typically rely on handcrafted, rule-based or double propagation methods in the opinion mining process which are known to be time-consuming and often inclined to errors. This could affect the reliability and performance of the recommender systems (RS). Therefore, in this paper, we propose a weighted Aspect-based Opinion mining using Deep learning method for Recommender system (AODR) that can extract product's aspects and the underlying weighted user opinions from the review text using a deep learning method and then fuse them into extended collaborative filtering (CF) technique for improving the RS. The proposed method is basically comprised of two components: (1) Aspect-based opinion mining module which aims to extract the product aspects from the review text to generate aspect rating matrix. (2) Recommendation generation component that uses tensor factorization (TF) technique to compute weighted aspect ratings and finally infer the overall rating prediction. We evaluate the proposed model in terms of both aspect extraction and recommendation performance. Experiment results on different datasets show that our AODR model achieves better results compared to the baselines.

Temporal‐aware and sparsity‐tolerant hybrid collaborative recommendation method with privacy preservation

SummaryWith the explosive growth of cloud services, how to design effective recommendation models has become more and more important. Temporal information has been proved to be an important factor affecting recommendation performance. Actually, both of user behaviors and QoS performance of services are time‐sensitive, especially in dynamic cloud environment. However, most existing collaborative recommendation methods seldom consider temporal influence to QoS performance. Furthermore, with the ever‐increasing number of security threats in clouds, privacy preservation becomes an important problem to be addressed in recommender systems. Based on these observations, in this paper, we propose a temporal‐aware and sparsity‐tolerant hybrid collaborative recommendation method with privacy preservation, where tensor factorization‐based CF model is integrated into neighborhood‐based CF model to achieve improved recommendation performance. Specifically, temporal influence is considered into recommendation model by distinguishing short‐term QoS metrics from long‐term QoS metrics. A privacy‐aware time aggregation mechanism is adopted to preserve the sensitive time information of users. Moreover, to deal with the sparsity problem, a sparsity‐oriented data QoS rating prediction mechanism based on tensor factorization technique is applied to predict the missing or unrated QoS rating data. Finally, predictions are made based on both stable and temporal nearest neighbors. Experiments are conducted to demonstrate the prediction performance of our proposal.

Tensor Based Blind Source Separation in Longitudinal Magnetic Resonance Imaging Analysis.

Study of white matter (WM) fiber-bundles is a crucial challenge in the investigation of neurological diseases like multiple sclerosis (MS). In this activity, the amount of data to process is huge, and an automated approach to carry out this task is in order.In this paper we show how tensor-based blind source separation (BSS) techniques can be successfully applied to model complex anatomical brain structures. More in detail, we show how through vector hankelization it is possible to formalize data extracted from WM fiber-bundles using a tensor model. Two main tensor factorization techniques, namely (Lr, Lr, 1) block term decomposition (BTD) and canonical polyadic decomposition (CPD), were applied to the generated tensor. The information extracted from the factorization was then used to differentiate between sets of fibers, within the bundle, affected by the pathology and normal appearing fibers.Performances of the proposed tensor-based model was evaluated on simulated data representing pathological effects of MS. Results show the capability of our tensor-based model to detect small pathological phenomena appearing along WM fibers.

Convolutional Recurrent Neural Network-Based Event Detection in Tunnels Using Multiple Microphones

This paper proposes a sound event detection (SED) method in tunnels to prevent further uncontrollable accidents. Tunnel accidents are accompanied by crashes and tire skids, which usually produce abnormal sounds. Since the tunnel environment always has a severe level of noise, the detection accuracy can be greatly reduced in the existing methods. To deal with the noise issue in the tunnel environment, the proposed method involves the preprocessing of tunnel acoustic signals and a classifier for detecting acoustic events in tunnels. For preprocessing, a non-negative tensor factorization (NTF) technique is used to separate the acoustic event signal from the noisy signal in the tunnel. In particular, the NTF technique developed in this paper consists of source separation and online noise learning. In other words, the noise basis is adapted by an online noise learning technique for enhancement in adverse noise conditions. Next, a convolutional recurrent neural network (CRNN) is extended to accommodate the contributions of the separated event signal and noise to the event detection; thus, the proposed CRNN is composed of event convolution layers and noise convolution layers in parallel followed by recurrent layers and the output layer. Here, a set of mel-filterbank feature parameters is used as the input features. Evaluations of the proposed method are conducted on two datasets: a publicly available road audio events dataset and a tunnel audio dataset recorded in a real traffic tunnel for six months. In the first evaluation where the background noise is low, the proposed CRNN-based SED method with online noise learning reduces the relative recognition error rate by 56.25% when compared to the conventional CRNN-based method with noise. In the second evaluation, where the tunnel background noise is more severe than in the first evaluation, the proposed CRNN-based SED method yields superior performance when compared to the conventional methods. In particular, it is shown that among all of the compared methods, the proposed method with the online noise learning provides the best recognition rate of 91.07% and reduces the recognition error rates by 47.40% and 28.56% when compared to the Gaussian mixture model (GMM)–hidden Markov model (HMM)-based and conventional CRNN-based SED methods, respectively. The computational complexity measurements also show that the proposed CRNN-based SED method requires a processing time of 599 ms for both the NTF-based source separation with online noise learning and CRNN classification when the tunnel noisy signal is one second long, which implies that the proposed method detects events in real-time.

Pre-processing the nuclear many-body problem

The solution of the nuclear A-body problem encounters severe limitations from the size of many-body operators. These limitations are typically related to both the (iterative) storing of the associated tensors and to the computational time related to their multiple contractions in the calculation of various quantities of interest. However, not all the degrees of freedom encapsulated into these tensors equally contribute to the description of many-body observables. Identifying systematic and dominating patterns, a relevant objective is to achieve an \emph{a priori} reduction to the most relevant degrees of freedom via a pre-processing of the A-body problem. The present paper is dedicated to the analysis of two different paradigms to do so. The factorization of tensors in terms of lower-rank ones, whose know-how has been recently transferred to the realm of nuclear structure, is compared to a reduction of the tensors' index size based on an importance truncation. While the objective is to eventually utilize these pre-processing tools in the context of non-perturbative many-body methods, benchmark calculations are presently performed within the frame of perturbation theory. More specifically, we employ the recently introduced Bogoliubov many-body perturbation theory that is systematically applicable to open-shell nuclei displaying strong correlations. This extended perturbation theory serves as a jumpstart for non-perturbative Bogoliubov coupled cluster and Gorkov self-consistent Green's function theories. Results obtained in "small" model spaces are equally encouraging for tensor factorization and importance truncation techniques. While the former requires significant numerical developments to be applied in large model spaces, the latter is presently applied in this context and demonstrates great potential to enable high-accuracy calculations at a much reduced computational cost.

Tensor-decomposition techniques for ab initio nuclear structure calculations: From chiral nuclear potentials to ground-state energies

The impact of applying state-of-the-art tensor factorization techniques to modern nuclear Hamiltonians derived from chiral effective field theory is investigated. Subsequently, the error induced by the tensor decomposition of the input Hamiltonian on ground-state energies of closed-shell nuclei calculated via second-order many-body perturbation theory is benchmarked. With the aid of the factorized Hamiltonian, the second-order perturbative correction to ground-state energies is decomposed and the scaling properties of the underlying tensor network are discussed. The employed tensor formats are found to lead to an efficient data compression of two-body matrix elements of the nuclear Hamiltonian. In particular, the sophisticated \emph{tensor hypercontraction} (THC) scheme yields low tensor ranks with respect to both harmonic-oscillator and Hartree-Fock single-particle bases. It is found that the tensor rank depends on the two-body total angular momentum $J$ for which one performs the decomposition, which is itself directly related to the sparsity the corresponding tensor. Furthermore, including normal-ordered two-body contributions originating from three-body interactions does not compromise the efficient data compression. Ultimately, the use of factorized matrix elements authorizes controlled approximations of the exact second-order ground-state energy corrections. In particular, a small enough error is obtained from low-rank factorizations in $^{4}$He, $^{16}$O and $^{40}$Ca.

Block Term Decomposition of ECG Recordings for Atrial Fibrillation Analysis: Temporal and Inter-Patient Variability

Responsible for 25% of strokes and 1/3 of hospitalizations due to cardiac related disturbances, atrial fibrillation (AF) is the most common sustained cardiac arrhythmia in clinical practice, considered as the last great frontier of cardiac electrophysiology. Its mechanisms are not completely understood, and a precise analysis of the atrial activity (AA) signal in electrocardiogram (ECG) recordings is necessary to better understand this challenging cardiac condition. Recently, the block term decomposition (BTD) has been proposed as a powerful tool to noninvasively extract AA in AF ECG signals. However, this tensor factorization technique was performed only in short ECG recordings and illustrated in single patients. To assess its performance and variability through different subjects, BTD is applied to a population of 10 AF patients in this paper. Also, its time variability is evaluated by means of long segments of AF ECG with varying observation window size. Experimental results show the consistency of BTD as an AA extraction tool, outperforming two well-known matrix-based methods in most of the observed cases for long and short AF ECG recordings.

Personalized Time-Aware Tag Recommendation

Personalized tag recommender systems suggest a list of tags to a user when he or she wants to annotate an item. They utilize users’ preferences and the features of items. Tensorfactorization techniques have been widely used in tag recommendation. Given the user-item pair, although the classic PITF (Pairwise Interaction Tensor Factorization) explicitly models the pairwise interactions among users, items and tags, it overlooks users’ short-term interests and suffers from data sparsity. On the other hand, given the user-item-time triple, time-aware approaches like BLL (Base-Level Learning) utilize the time effect to capture the temporal dynamics and the most popular tags on items to handle cold start situation of new users. However, it works only on individual level and the target resource level, which cannot find users’ potential interests. In this paper, we propose an unified tag recommendation approach by considering both time awareness and personalization aspects, which extends PITF by adding weightsto user-tag interaction and item-tag interaction respectively. Compared to PITF, our proposed model can depict temporal factor by temporal weights and relieve data sparsity problem by referencing the most popular tags on items. Further, our model brings collaborative filtering (CF) to time-aware models, which can mine information from global data and help improving the ability of recommending new tags. Different from the power-form functions used in the existing time aware recommendation models, we use the Hawkes process with the exponential intensity function to improve the model’s efficiency. The experimental results show that our proposed model outperforms the state of the art tag recommendation methods in accuracy and has better ability to recommend new tags.

Matrix Factorization Techniques for Context-Aware Collaborative Filtering Recommender Systems: A Survey

Collaborative Filtering Recommender Systems predict user preferences for online information, products or services by learning from past user-item relationships. A predominant approach to Collaborative Filtering is Neighborhood-based, where a user-item preference rating is computed from ratings of similar items and/or users. This approach encounters data sparsity and scalability limitations as the volume of accessible information and the active users continue to grow leading to performance degradation, poor quality recommendations and inaccurate predictions. Despite these drawbacks, the problem of information overload has led to great interests in personalization techniques. The incorporation of context information and Matrix and Tensor Factorization techniques have proved to be a promising solution to some of these challenges. We conducted a focused review of literature in the areas of Context-aware Recommender Systems utilizing Matrix Factorization approaches. This survey paper presents a detailed literature review of Context-aware Recommender Systems and approaches to improving performance for large scale datasets and the impact of incorporating contextual information on the quality and accuracy of the recommendation. The results of this survey can be used as a basic reference for improving and optimizing existing Context-aware Collaborative Filtering based Recommender Systems. The main contribution of this paper is a survey of Matrix Factorization techniques for Context-aware Collaborative Filtering Recommender Systems.