Tensor Factorization Model Research Articles

A Bayesian tensor ring decomposition model with automatic rank determination for spatiotemporal traffic data imputation

Recently, tensor factorization models have shown superiority in solving traffic data imputation problem. However, these approaches have a limited ability to learn traffic data correlations and are easy to overfit when the pre-defined rank is large and the available data is limited. In this paper, we propose a Bayesian tensor ring decomposition model, utilizing Variational Bayesian Inference to solve the model. Firstly, tensor ring decomposition with an enhanced representational capability is used to decompose partially observed data into factor tensors to capture the correlation in traffic data. Secondly, to address the issue of selecting large pre-defined rank when data availability is limited, an automatic determination mechanism of tensor ring ranks is proposed. This mechanism can be implemented by pruning the zero-component horizontal and frontal slices of the core factors in each iteration, reducing the dimensions of the core factors and consequently lowering the tensor ring ranks. Finally, extensive experiments on synthetic data and four diverse types of real-world traffic datasets demonstrate the superiority of the proposed model. In the Guangzhou dataset, the maximum improvement in Mean Absolute Percentage Error can reach 15 % compared to the most competitive baseline model.

CTF-DDI: Constrained tensor factorization for drug–drug interactions prediction

Computational approaches for predicting drug–drug interactions (DDI) can significantly facilitate combination therapy and drug discovery. Existing similarity-based methods often overlook simple yet valuable structural information or ignore multiple relationships from biological entities (e.g., target proteins and enzymes). Meanwhile, matrix factorization-based methods can alleviate the inherent sparsity issues in DDI data. However, this line of work usually only considers the original association information of DDI pairs. To address these issues, we proposed a novel tensor factorization strategy with effective constraint terms (CTF-DDI) for potential DDI prediction. Specifically, we first obtained drug features by constructing specific similarity matrices based on drug structure and drug-related biological associations. Then, a novel constrained tensor factorization (CTF) module was designed to further reconstruct drug similarity by introducing Hessian and L2,1 regularization as constraints. Finally, we trained a deep neural network to extract nonlinear features for DDI prediction. Experimental results on two benchmark datasets demonstrated that the proposed CTF-DDI model outperforms classical tensor factorization and deep learning models. Furthermore, ablation and case studies validated the performance of CTF-DDI in DDI prediction. The source code of CTF-DDI is available at https://github.com/angelfacedac/CTF_DDI.

CTITF: A tensor factorization model with constrained bidirectional user trust and implicit feedback for context-aware recommender systems

Recommender systems offer an efficient solution to the problem of information overload, which is exacerbated by the rapid expansion of data. Context-aware recommender systems (CARS) have become prominent, incorporating contextual information to provide more desirable recommendations. While the dimension of contextual information rises, the complexity of algorithms increases as well, highlighting the data sparsity problem. In this paper, we propose CTITF, a novel tensor factorization model that incorporates constrained user bidirectional trust and implicit feedback. First, CTITF distinguishes the roles of trust networks in terms of the directionality of trust relationships: the trustor and the trustee. CTITF then constructs trust influences and biases for both roles to linearly represent user preferences. Furthermore, CTITF improves the rating prediction function and recommendation objective function based on CP factorization. Finally, CTITF filters out trust relationships where users do not have the same history items. It also uses contextual information to limit the amount of implicit feedback, addressing the issue of high time complexity. Experiments on two real-world datasets illustrate that the proposed model outperforms previous TF-based models and social recommender systems in terms of rating prediction accuracy. In addition, the CTITF model reduces the training time by 50.28% compared to the trustTF model, which also uses trust.

Multi-faceted analysis and prediction for the outbreak of pediatric respiratory syncytial virus.

Respiratory syncytial virus (RSV) is a significant cause of pediatric hospitalizations. This article aims to utilize multisource data and leverage the tensor methods to uncover distinct RSV geographic clusters and develop an accurate RSV prediction model for future seasons. This study utilizes 5-year RSV data from sources, including medical claims, CDC surveillance data, and Google search trends. We conduct spatiotemporal tensor analysis and prediction for pediatric RSV in the United States by designing (i) a nonnegative tensor factorization model for pediatric RSV diseases and location clustering; (ii) and a recurrent neural network tensor regression model for county-level trend prediction using the disease and location features. We identify a clustering hierarchy of pediatric diseases: Three common geographic clusters of RSV outbreaks were identified from independent sources, showing an annual RSV trend shifting across different US regions, from the South and Southeast regions to the Central and Northeast regions and then to the West and Northwest regions, while precipitation and temperature were found as correlative factors with the coefficient of determination R2≈0.5, respectively. Our regression model accurately predicted the 2022-2023 RSV season at the county level, achieving R2≈0.3 mean absolute error MAE < 0.4 and a Pearson correlation greater than 0.75, which significantly outperforms the baselines with P-values <.05. Our proposed framework provides a thorough analysis of RSV disease in the United States, which enables healthcare providers to better prepare for potential outbreaks, anticipate increased demand for services and supplies, and save more lives with timely interventions.

A many‐objective optimization‐based local tensor factorization model for skin cancer detection

SummaryExploring the associations between microRNAs (miRNAs) and diseases can identify potential disease features. Prediction of miRNA‐skin cancer associations has become an effective method for detecting skin cancer, as direct detection of skin cancer is challenging. However, traditional binary associations prediction methods overlook high‐order feature differences of miRNAs in different types of skin cancer. Although current tensor decomposition methods have addressed this, they assume a consistent composition standard of global tensor elements, ignoring differences in the composition standards of local sub‐tensors. This leads to poor prediction accuracy and comprehensiveness and fails to consider the credibility and diversity requirements of patients and doctors in practical applications. In this paper, we represent miRNA‐skin cancer associations as a tensor and employ tensor decomposition for tensor completion to achieve prediction purposes. First, we propose a credibility evaluation indicator and introduce four objective functions: accuracy, comprehensiveness, credibility, and diversity, to construct a many‐objective local tensor factorization model (MOLTF). Then, to avoid wrong individuals when solving this model with genetic algorithms, we propose a corrected single‐point crossover operator and a corrected multi‐point mutation operator. On the miRNA‐disease dataset HMDD v3.2, our algorithm improves accuracy by 14.5% compared to the recent baseline, demonstrating its effectiveness.

Context-Aware Point-of-Interest Recommendation Based on Similar User Clustering and Tensor Factorization

The rapid development of big data technology and mobile intelligent devices has led to the development of location-based social networks (LBSNs). To understand users’ behavioral patterns and improve the accuracy of location-based services, point-of-interest (POI) recommendation has become an important task. In contrast to the general task of product recommendation, POI recommendation faces the problems of the sparsity and weak semantics of user check-in data. To address these issues, an increasing number of studies have improved the accuracy of POI recommendations by introducing contextual information such as geographical, temporal, textual, and social relations. However, the rich context also brings great challenges to POI recommendation, such as the low utilization rate of context information, difficulty in balancing the richness of contextual information, and the complexity of the recommendation matrix. Considering that similar users have more interest preferences in common than users generally have, the check-in information of similar users has greater reference meaning. Thus, we propose a personalized POI recommendation method named CULT-TF, which incorporates similar users’ contextual information into the tensor factorization model. First, we present a user activity model and a user similarity model, which integrate contextual information to calculate the user activity and similarity between users. According to user activity, the most representative active users are selected as user clustering centers, and then users are clustered based on user similarity into several similar user clusters (C). Next, we construct a third-order tensor (user-location-time matrix) for each user cluster by using user activity, POI popularity, and time slot popularity as the eigenvalues in the user (U), location (L), and time (T) dimensions, and the eigenvalue of each dimension is modeled by integrating contextual information of users’ check-in behavior at the user, location, and time levels. Similar user clustering reduces the number of users in tensor modeling, reducing the U dimension. To further reduce the complexity of the recommendation matrix, the reduction of the L dimension is achieved through ROI (region of interest) clustering, and the reduction of the T dimension is achieved through time slot encoding. Then, we use tensor factorization (TF) to obtain the recommendation results. Our method decreases the complexity of the tensor matrix and integrates rich contextual information on users’ check-in behavior. Finally, we conducted a comprehensive performance evaluation of CULT-TF using real-world LBSN datasets from Brightkite. The experimental results show that our proposed method performs much better than other recommendation methods in terms of precision and recall.

Regularized online tensor factorization for sparse knowledge graph embeddings

Knowledge Graphs represent real-world facts and are used in several applications; however, they are often incomplete and have many missing facts. Link prediction is the task of completing these missing facts from existing ones. Embedding models based on Tensor Factorization attain state-of-the-art results in link prediction. Nevertheless, the embeddings they produce can not be easily interpreted. Inspired by previous work on word embeddings, we propose inducing sparsity in the bilinear tensor factorization model, RESCAL, to build interpretable Knowledge Graph embeddings. To overcome the difficulties that stochastic gradient descent has when producing sparse solutions, we add l_1 regularization to the learning objective by using the generalized Regularized Dual Averaging online optimization algorithm. The proposed method substantially improves the interpretability of the learned embeddings while maintaining competitive performance in the standard metrics.

Low-tubal-rank tensor factorization on constant curvature Riemann manifold with mixture of Gaussians

Tensor factorization for recovering a tensor has been applied extensively in image processing. The existing tensor factorization methods have shown effective performance, however, they ignore that the plenty of low-rank components of the tensors in the real scenarios are manifold-valued. To capture more precise low-rank components, we propose a tensor factorization model with a low-rank subspace representation on the constant curvature Riemann manifold. In addition, to make the tensor factorization method more capable of adaptive complex noise, we model the noise on each mode of a tensor with a specific mixture of Gaussians (MoG). The experiments on images and videos demonstrate that the proposed method outperforms the state-of-the-art methods.

Fast weighted CP decomposition for context-aware recommendation with explicit and implicit feedback

As an effective method for high-dimensional information processing, tensor factorization has been widely used in context-aware recommendation. However, the multi-dimensional processing of tensor factorization usually leads to more serious data sparsity. Making full use of the data in recommendation systems, including a small amount of explicit rating data and a large amount of implicit feedback data such as clicks, plays, and purchases, can effectively alleviate this problem. Meanwhile, the processing of sparse explicit data and huge implicit feedback data makes tensor factorization face a severe challenge of computational efficiency. Therefore, this study proposes a fast tensor factorization model named FWCP that unifies explicit and implicit feedback information, which mainly studies as follows: (1) Assigning non-uniform weights to observed and unobserved values to unify explicit and implicit feedback information; (2) Memorizing intermediate data which generated by element-wise alternating least squares(ALS), and implementing parallel optimization of FWCP, which can reduce the training time of tensor factorization based on whole-data. The evaluation results on two real datasets showed that FWCP proposed in this study is superior to other context-aware recommendation algorithms.

A Probit Tensor Factorization Model For Relational Learning

With the proliferation of knowledge graphs, modeling data with complex multi-relational structure has gained increasing attention in the area of statistical relational learning. One of the most important goals of statistical relational learning is link prediction, that is, predicting whether certain relations exist in the knowledge graph. A large number of models and algorithms have been proposed to perform link prediction, among which tensor factorization method has proven to achieve state-of-the-art performance in terms of computation efficiency and prediction accuracy. However, a common drawback of the existing tensor factorization models is that the missing relations and nonexisting relations are treated in the same way, which results in a loss of information. To address this issue, we propose a binary tensor factorization model with probit link, which not only inherits the computation efficiency from the classic tensor factorization model but also accounts for the binary nature of relational data. Our proposed probit tensor factorization (PTF) model shows advantages in both the prediction accuracy and interpretability. Supplementary files for this article are available online.

Dictionary-Based Low-Rank Approximations and the Mixed Sparse Coding Problem

Constrained tensor and matrix factorization models allow to extract interpretable patterns from multiway data. Therefore crafting efficient algorithms for constrained low-rank approximations is nowadays an important research topic. This work deals with columns of factor matrices of a low-rank approximation being sparse in a known and possibly overcomplete basis, a model coined as Dictionary-based Low-Rank Approximation (DLRA). While earlier contributions focused on finding factor columns inside a dictionary of candidate columns, i.e., one-sparse approximations, this work is the first to tackle DLRA with sparsity larger than one. I propose to focus on the sparse-coding subproblem coined Mixed Sparse-Coding (MSC) that emerges when solving DLRA with an alternating optimization strategy. Several algorithms based on sparse-coding heuristics (greedy methods, convex relaxations) are provided to solve MSC. The performance of these heuristics is evaluated on simulated data. Then, I show how to adapt an efficient MSC solver based on the LASSO to compute Dictionary-based Matrix Factorization and Canonical Polyadic Decomposition in the context of hyperspectral image processing and chemometrics. These experiments suggest that DLRA extends the modeling capabilities of low-rank approximations, helps reducing estimation variance and enhances the identifiability and interpretability of estimated factors.

Improving Sales Forecasting Accuracy: A Tensor Factorization Approach with Demand Awareness

Because of the accessibility of big data collections from consumers, products, and stores, advanced sales forecasting capabilities have drawn great attention from many businesses, especially those in retail, because of the importance of forecasting in decision making. Improvement of forecasting accuracy, even by a small percentage, may have a substantial impact on companies’ production and financial planning, marketing strategies, inventory controls, and supply chain management. Specifically, our research goal is to forecast the sales of each product in each store in the near future. Motivated by tensor factorization methodologies for context-aware recommender systems, we propose a novel approach called the advanced temporal latent factor approach to sales forecasting, or ATLAS for short, which achieves accurate and individualized predictions for sales by building a single tensor factorization model across multiple stores and products. Our contribution is a combination of a tensor framework (to leverage information across stores and products), a new regularization function (to incorporate demand dynamics), and extrapolation of the tensor into future time periods using state-of-the-art statistical (seasonal autoregressive integrated moving-average models) and machine-learning (recurrent neural networks) models. The advantages of ATLAS are demonstrated on eight product category data sets collected by Information Resources, Inc., where we analyze a total of 165 million weekly sales transactions of over 15,560 products from more than 1,500 grocery stores.Summary of Contribution: Sales forecasting has been a task of long-standing importance. Accurate sales forecasting provides critical managerial implications for companies’ decision making and operations. Improvement of forecasting accuracy may have a substantial impact on companies’ production planning, marketing strategies, inventory controls, and supply chain management, among other things. This paper proposes a novel computational (machine-learning-based) approach to sales forecasting and thus is positioned directly at the intersection of computing and business/operations research.

Large scale tensor regression using kernels and variational inference

We outline an inherent flaw of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this. We coin our methodology kernel fried tensor (KFT) and present it as a large-scale prediction and forecasting tool for high dimensional data. Our results show superior performance against LightGBM and Field aware factorization machines (FFM), two algorithms with proven track records, widely used in large-scale prediction. We also develop a variational inference framework for KFT which enables associating the predictions and forecasts with calibrated uncertainty estimates on several datasets.

A Temporal Data-Driven Player Model for Dynamic Difficulty Adjustment

Many computer games of all genres pit the player against a succession of increasingly difficult challenges such as combat with computer-controlled enemies and puzzles. Part of the fun of computer games is to master the skills necessary to complete the game. Challenge tailoring is the problem of matching the difficulty of skill-based events over the course of a game to a specific player's abilities. We present a tensor factorization approach to predicting player performance in skill-based computer games. Our tensor factorization approach is data-driven and can predict changes in players' skill mastery over time, allowing more accurate tailoring of challenges. We demonstrate the efficacy and scalability of tensor factorization models through an empirical study of human players in a simple role-playing combat game. We further find a significant correlation between these performance ratings and player subjective experiences of difficulty and discuss ways our model can be used to optimize player enjoyment.

A parameter-less algorithm for tensor co-clustering

The majority of the data produced by human activities and modern cyber-physical systems involve complex relations among their features. Such relations can be often represented by means of tensors, which can be viewed as generalization of matrices and, as such, can be analyzed by using higher-order extensions of existing machine learning methods, such as clustering and co-clustering. Tensor co-clustering, in particular, has been proven useful in many applications, due to its ability of coping with n-modal data and sparsity. However, setting up a co-clustering algorithm properly requires the specification of the desired number of clusters for each mode as input parameters. This choice is already difficult in relatively easy settings, like flat clustering on data matrices, but on tensors it could be even more frustrating. To face this issue, we propose a new tensor co-clustering algorithm that does not require the number of desired co-clusters as input, as it optimizes an objective function based on a measure of association across discrete random variables (called Goodman and Kruskal’s tau) that is not affected by their cardinality. We introduce different optimization schemes and show their theoretical and empirical convergence properties. Additionally, we show the effectiveness of our algorithm on both synthetic and real-world datasets, also in comparison with state-of-the-art co-clustering methods based on tensor factorization and latent block models.

SWIFT: Scalable Wasserstein Factorization for Sparse Nonnegative Tensors

Existing tensor factorization methods assume that the input tensor follows some specific distribution (i.e. Poisson, Bernoulli, and Gaussian), and solve the factorization by minimizing some empirical loss functions defined based on the corresponding distribution. However, it suffers from several drawbacks: 1) In reality, the underlying distributions are complicated and unknown, making it infeasible to be approximated by a simple distribution. 2) The correlation across dimensions of the input tensor is not well utilized, leading to sub-optimal performance. Although heuristics were proposed to incorporate such correlation as side information under Gaussian distribution, they can not easily be generalized to other distributions. Thus, a more principled way of utilizing the correlation in tensor factorization models is still an open challenge. Without assuming any explicit distribution, we formulate the tensor factorization as an optimal transport problem with Wasserstein distance, which can handle non-negative inputs. We introduce SWIFT, which minimizes the Wasserstein distance that measures the distance between the input tensor and that of the reconstruction. In particular, we define the N-th order tensor Wasserstein loss for the widely used tensor CP factorization and derive the optimization algorithm that minimizes it. By leveraging sparsity structure and different equivalent formulations for optimizing computational efficiency, SWIFT is as scalable as other well-known CP algorithms. Using the factor matrices as features, SWIFT achieves up to 9.65% and 11.31% relative improvement over baselines for downstream prediction tasks. Under the noisy conditions, SWIFT achieves up to 15% and 17% relative improvements over the best competitors for the prediction tasks.

Urban Noise Inference Model Based on Multiple Views and Kernel Tensor Decomposition

Prediction of urban noise is becoming more significant for tackling noise pollution and protecting human mental health. However, the existing noise prediction algorithms neglected not only the correlation between noise regions, but also the nonlinearity and sparsity of the data, which resulted in low accuracy of filling in the missing entries of data. In this paper, we propose a model based on multiple views and kernel-matrix tensor decomposition to predict the noise situation at different times of day in each region. We first construct a kernel tensor decomposition model by using kernel mapping in order to speed decomposition rate and realize stable estimate the prediction system. Then, we analyze and compute the cause of the noise from multiple views including computing the similarity of regions and the correlation between noise categories by kernel distance, which improves the credibility to infer the noise situation and the categories of regions. Finally, we devise a prediction algorithm based on the kernel-matrix tensor factorization model. We evaluate our method with a real dataset, and the experiments to verify the advantages of our method compared with other existing baselines.

Ontology Guided Sparse Tensor Factorization for joint recommendation with hierarchical relationships

Although recommender systems enjoy widespread adoption in numerous different production settings, standard methods draw only on previous purchases or ratings, and optionally simple customer or product features. In many domains, however, the purchase or rating history is very sparse. Standard approaches suffer from such data sparsity and neglect to account for important additional dependencies that can be taken into consideration. This motivates us to design a recommendation model with the ability to exploit hierarchical relationships such as product series, manufacturers, or even suppliers. To this end, we propose our Ontology Guided Multi-Relational Tensor Factorization model, which models such connections via a multilevel tree structure. To solve the challenging optimization problem, we develop an efficient iterative algorithm relying on Moreau-Yosida regularization and analyzed the complexity. On real-world data crawled from automobile-related websites, we find that the proposed model outperforms state-of-the-art methods.

TBTF: an effective time-varying bias tensor factorization algorithm for recommender system

Context-aware processing is a research hotspot in the recommendation area, which achieves better recommendation accuracy by considering more context information such as time, location and etc. besides the information of the users, items and ratings. Tensor factorization is an effective algorithm in context-aware recommendation and current approaches show that adding bias to the tensor factorization model can improve the accuracy. However, users’ rating preferences fluctuate greatly over time, which makes bias fluctuate with time too. Current context-aware recommendation algorithms ignore this problem, and usually use the same bias for a user or an item in different time. Aiming at this problem, this paper first considers the time-varying effect on user bias and item bias in context-aware recommendation, and proposes a time-varying bias tensor factorization recommendation algorithm based on the bias tensor factorization model (BiasTF). We experiment on two real datasets, and the experimental results show that the proposed algorithms get better accuracy than other algorithms.

Integrating probabilistic tensor factorization with Bayesian supervised learning for dynamic ridesharing pattern analysis

In the era of transportation big data, the analysis of mobility patterns generally involves large quantities of datasets with high-dimensional variables recording individual travelers’ activities and socio-economic attributes, bringing new challenges to researchers. Conventional regression-based models commonly require complex structures in depicting random or fixed effects with a considerable number of parameters to estimate, and state-of-the-art machine learning models are regarded as black-boxes that are not clear in interpreting the mechanism in human mobility. To overcome the challenges of capturing complex high-order relationships among variables of interest, this paper proposes a Bayesian supervised learning tensor factorization (BSTF) model for the classification of travel choices in the mobility pattern analysis. The BSTF model induces a hierarchical probabilistic structure between predictor variables and the dependent variable, which offers a nature supervised learning foundation via Bayesian inference. Latent class (LC) variables are considered in the BSTF model to discover hidden preferences/states among travelers associated with their mobility patterns. We apply the BSTF model to analyze passenger-side choice patterns between diverse service options on a ride-sourcing platform, drawing increasing attention during recent years. A case study with a real-world dynamic ridesharing dataset in Hangzhou, China, is conducted. Different cases of training sizes are utilized to fit the proposed BSTF model as well as some other state-of-the-art machine learning models. By identifying significant variables and derive their probabilistic relationship between service types (i.e., ridesharing, non-sharing, and taxi), the proposed BSTF model offers good performance in both classification accuracy and the interpretability of shared mobility.