Field-aware Factorization Machines Research Articles

Large scale tensor regression using kernels and variational inference

We outline an inherent flaw of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this. We coin our methodology kernel fried tensor (KFT) and present it as a large-scale prediction and forecasting tool for high dimensional data. Our results show superior performance against LightGBM and Field aware factorization machines (FFM), two algorithms with proven track records, widely used in large-scale prediction. We also develop a variational inference framework for KFT which enables associating the predictions and forecasts with calibrated uncertainty estimates on several datasets.

Synergistic drug combinations prediction by integrating pharmacological data

There is compelling evidence that synergistic drug combinations have become promising strategies for combating complex diseases, and they have evident predominance comparing to traditional one drug - one disease approaches. In this paper, we develop a computational method, namely SyFFM, that takes pharmacological data into consideration and applies field-aware factorization machines to analyze and predict potential synergistic drug combinations. Firstly, features of drug pairs are constructed based on associations between drugs and target, and enzymes, and indication areas. Then, the synergistic scores of drug combinations are obtained by implementing field-aware factorization machines on latent vector space of these features. Finally, synergistic combinations can be predicted by introducing a threshold. We applied SyFFM to predict pairwise synergistic combinations and three-drug synergistic combinations, and the performance is good in terms of cross-validation. Besides, more than 90% combinations of the top ranked predictions are proved by literature and the analysis of parameters in model shows that our method can help to investigate and explain synergistic mechanisms underlying combinatorial therapy.

Differentiating Regularization Weights -- A Simple Mechanism to Alleviate Cold Start in Recommender Systems

Matrix factorization (MF) and its extended methodologies have been studied extensively in the community of recommender systems in the last decade. Essentially, MF attempts to search for low-ranked matrices that can (1) best approximate the known rating scores, and (2) maintain low Frobenius norm for the low-ranked matrices to prevent overfitting. Since the two objectives conflict with each other, the common practice is to assign the relative importance weights as the hyper-parameters to these objectives. The two low-ranked matrices returned by MF are often interpreted as the latent factors of a user and the latent factors of an item that would affect the rating of the user on the item. As a result, it is typical that, in the loss function, we assign a regularization weight λ p on the norms of the latent factors for all users, and another regularization weight λ q on the norms of the latent factors for all the items. We argue that such a methodology probably over-simplifies the scenario. Alternatively, we probably should assign lower constraints to the latent factors associated with the items or users that reveal more information, and set higher constraints to the others. In this article, we systematically study this topic. We found that such a simple technique can improve the prediction results of the MF-based approaches based on several public datasets. Specifically, we applied the proposed methodology on three baseline models -- SVD, SVD++, and the NMF models. We found that this technique improves the prediction accuracy for all these baseline models. Perhaps more importantly, this technique better predicts the ratings on the long-tail items, i.e., the items that were rated/viewed/purchased by few users. This suggests that this approach may partially remedy the cold-start issue. The proposed method is very general and can be easily applied on various recommendation models, such as Factorization Machines, Field-aware Factorization Machines, Factorizing Personalized Markov Chains, Prod2Vec, Behavior2Vec, and so on. We release the code for reproducibility. We implemented a Python package that integrates the proposed regularization technique with the SVD, SVD++, and the NMF model. The package can be accessed at https://github.com/ncu-dart/rdf.

Research on CTR Prediction Based on Deep Learning

Click-through rate (CTR) prediction is critical in Internet advertising and affects web publisher’s profits and advertiser’s payment. In the CTR prediction, the interaction between features is a key factor affecting the prediction rate. The traditional method of obtaining features using feature extraction did not consider the sparseness of advertising data and the highly nonlinear association between features. To reduce the sparseness of data and to mine the hidden features in advertising data, a method that learns the sparse features is proposed. Our method exploits dimension reduction based on decomposition and combines the power of field-aware factorization machines and deep learning to portray the nonlinear associated relationship of data to solve the sparse feature learning problem. The experiment shows that our method improves the effect of CTR prediction and produces economic benefits in Internet advertising.

Sentiment-aware personalized tweet recommendation through multimodal FFM

For realizing quick and accurate access to desired information and effective advertisements or election campaigns, personalized tweet recommendation is highly demanded. Since multimedia contents including tweets are tools for users to convey their sentiment, users’ interest in tweets is strongly influenced by sentiment factors. Therefore, successful personalized tweet recommendation can be realized if sentiment in tweets can be estimated. However, sentiment factors were not taken into account in previous works and the performance of previous methods may be limited. To overcome the limitation, a method for sentiment-aware personalized tweet recommendation through multimodal Field-aware Factorization Machines (FFM) is newly proposed in this paper. Successful personalized tweet recommendation becomes feasible through the following three contributions: (i) sentiment factors are newly introduced into personalized tweet recommendation, (ii) users’ interest is modeled by deriving multimodal FFM that enables collaborative use of multiple factors in a tweet, i.e., publisher, topic and sentiment factors, and (iii) the effectiveness of using sentiment factors as well as publisher and topic factors is clarified from results of experiments using real-world datasets related to worldwide hot topics, “#trump”, “#hillaryclinton” and “#ladygaga”. In addition to showing the effectiveness of the proposed method, the applicability of the proposed method to other tasks such as advertisement and social analysis is discussed as a conclusion and future work of this paper.