Bayesian Probabilistic Matrix Factorization Research Articles

Mashup-Oriented Web API Recommendation Via Full-Text Semantic Mining of Developer Requirements

API recommendation differs from conventional recommendation systems in that it integrates the characteristics of development requirements, mashup, and API, where there are two primary challenges, i.e., how to effectively mine the personalized development requirements of developers and the sparseness of interaction data. This research proposes a novel model framework, called Bayesian Probabilistic Matrix Factorization Model with Text Similarity and Adversarial Training (SAMF), to address these two issues. Utilizing natural language processing technology to extract the full-text semantics of requirements documents, mashup descriptions, and API descriptions, fully mining personalized development requirements, and contextual information of mashups and APIs, and calculating text similarity, is our fundamental idea. Simultaneously, the collaborative filtering method is used to mine user needs, mashup, and API information from historical data, and the obtained text similarity is added to the training as auxiliary information. Furthermore, adversarial training is further incorporated to supplement data in order to minimize data sparsity and enhance model robustness and generalization. To assess the performance of SAMF, we run thorough experiments that illustrate the efficacy of each module of the model and explain the influence of hyperparameter settings. Specifically, compared to the baseline, the experimental findings demonstrate that SAMF can basically achieve better performance.

A decentralized trust inference approach with intelligence to improve data collection quality for mobile crowd sensing

Mobile Crowd Sensing (MCS) has been recognized as a promising param to construct numerous applications by employing enormous workers to perceive and collect data. The quality of MCS depends on the quality of the data submitted by the workers. Therefore, there is an urgent need to gain the trust of workers. In this paper, we propose a Decentralized Trust Inference (DTI) approach to improve data collection quality for MCS, mainly including the following components: (a) A trust evaluation method is proposed to obtain trust baselines of different levels, which can be used to assess the trust of workers. In addition, a data filling scheme on the basis of Bayesian Probabilistic Matrix Factorization (BPMF) is adopted to fill data when there is no credible baseline in the region. (b) Based on the DTI approach, we propose a worker recruitment method according to the priority of trust and bid ratio. Then, by preferentially selecting reliable and low-bid workers, we can improve data quality and reduce costs. Finally, theoretical analysis and experimental results demonstrate the effectiveness of our proposed scheme.

An improved constrained Bayesian probabilistic matrix factorization algorithm

Given the increasing growth of the Web and consequently the growth of e-commerce, the application of recommendation systems becomes more and more extensive. A good recommendation algorithm can provide a better user experience. In the collaborative filtering algorithm recommendation system, many existing approaches to collaborative filtering can neither handle very large datasets nor easily deal with users who have very few ratings, this paper proposes an improved constrained Bayesian probability matrix factorization algorithm. The algorithm introduces a potential similarity constraint matrix for specific sparsely scored users to affect the user’s feature vector, and uses the Logistic function to express the nonlinear relationship of the potential factors, combined with the Markov chain Monte Carlo method for training. Finally, the data set is used for testing and comparative evaluation. This experiment proves that the algorithmic model can be efficiently trained using Markov chain Monte Carlo methods by applying them to the MovieLens and Netflix dataset. The experimental results show that the algorithm has better predictive performance and is suitable for solving the problem of sparse rating matrix of specific users.

Prediction of dissolved gas concentration in transformer oil considering data loss scenarios in power system

The trend prediction of dissolved gas concentration in transformer oil can provide basis for transformer fault diagnosis, which is of vital significance to the safe operation of power system. However, due to the inevitable malfunction of monitoring equipment, it is difficult to collect all needed data in actual operation scenarios. Therefore, a method for predicting dissolved gas concentration in transformer oil for data loss scenarios is proposed based on Bayesian probabilistic matrix factorization (BPMF) and gated recurrent unit (GRU) neural network. Firstly, aiming at the problem of data loss in actual monitoring of dissolved gas in oil, BPMF is used to fill in the missing data. Then, a GRU neural network model is established to predict the trend of dissolved gas concentration in oil. Finally, the hyperparameters of the prediction model are selected and optimized by Bayesian theory. The examples show that this method can effectively fill in the missing part of the measured data. Compared with traditional prediction methods, the proposed method has higher prediction accuracy.

Integration of an AI-Based Platform and Flipped Classroom Instructional Model

This article aims to evaluate a flipped class teaching model based on an AI (artificial intelligence) Language Learning Platform. The platform adopts mainstream collaborative filtering and BPMF (Bayesian Probabilistic Matrix Factorization) and big data management methods to provide students’ detailed performance analysis reports and push recommendation exercises. The researcher reconciled the flipped teaching model with the AI-based language learning platforming in an experimental class and conventionally in a control class. At the end of the semester, the researcher compared students’ performance as well as their attitudes toward the course between the two classes through a postexam paper and a questionnaire. Combining quantitative and qualitative methods, the survey shows that although the AI-based language learning mode failed to significantly increase students’ English listening scores, compared with the students in the traditional class, those receiving the new mode held a more positive attitude toward their English listening experience, especially in regard to learning interest, study autonomy, and class involvement. The flipped class mode not only relieved students’ partial cognitive processing workloads in class, but also boosted their confidence to engage in classroom discussion by reducing their nervousness through before-class preview. The AI-based language learning platform mainly performs monitoring and tutoring functions which facilitate the flipped class instruction. This study proposes a new flipped teaching model and proves it effective, indicating that for the flipped class learning mode to be successful, it is of importance to integrate face-to-face classroom instruction with the AI-technology-assisted online learning experience in a way that they can coherently support each other.

Recommender systems for Personalized Work Instructions

The introduction of digital systems in manufacturing has increased the number of ways that assistant information reaches the human operators in assembly work spaces. Such digital systems can accustom to different degrees of support, based on the level of detail of the information and the used medium. As the production environment becomes increasingly flexible and the tasks are changing in a faster pace, a more agile way for recommendations of support or instructions is needed. This work will present and compare two ways to represent instructional support, namely, on-screen digital work instructions (DWI) and Augmented Reality (AR)- based instructions. Furthermore, in order to assist an operator with the appropriate instructional support, a recommender system (RS) is developed, which matches an operator’s proficiency, a task’s requirement and a level of instruction. A purely data-driven RS using Bayesian Probabilistic Matrix Factorization (BPMF) is compared with a skill-driven approach. The latter has the advantage that the recommended level of instruction is directly relatable to a skill-based operator proficiency metric. For the evaluation of the two instruction delivery approaches and for training and testing the RS, a group of 13 users were called to complete two industrial setups, alternating between the different instruction types. Both RS were able to outperform a naive baseline.

A Robust Bayesian Probabilistic Matrix Factorization Model for Collaborative Filtering Recommender Systems Based on User Anomaly Rating Behavior Detection

A Robust Bayesian Probabilistic Matrix Factorization Model for Collaborative Filtering Recommender Systems Based on User Anomaly Rating Behavior Detection

Prediction of Potential Associations Between MicroRNA and Disease Based on Bayesian Probabilistic Matrix Factorization Model.

The association between microRNAs (miRNAs) and diseases is significant to understand the development and progression of many human diseases. Given the cost and complexity of biological experiments, the computational method for predicting the potential association between miRNAs and disease will be an effective complement. In this article, we have developed a model (microRNA and disease based on Bayesian probabilistic matrix factorization, MDBPMF) based on a fully Bayesian treatment of the probabilistic matrix factorization to find potential associations between miRNAs and diseases by using the HMDDv2.0 database, which contains proven miRNA-disease associations. We show that Bayesian probabilistic matrix factorization models can be efficiently trained using Markov chain Monte Carlo methods by applying them to the HMDDv2.0 database. MDBPMF achieves reliable prediction with an average area under receiver operating characteristic curve of 0.8755 for eight complex diseases based on fivefold cross-validation, which indeed outperforms the state-of-the-art method. In addition, a case study of lung cancer further verifies the utility of our method.

A Bayesian tensor decomposition approach for spatiotemporal traffic data imputation

The missing data problem is inevitable when collecting traffic data from intelligent transportation systems. Previous studies have shown the advantages of tensor completion-based approaches in solving multi-dimensional data imputation problems. In this paper, we extend the Bayesian probabilistic matrix factorization model by Salakhutdinov and Mnih (2008) to higher-order tensors and apply it for spatiotemporal traffic data imputation tasks. In doing so, we care about not only the model configuration but also the representation of data (i.e., matrix, third-order tensor and fourth-order tensor). Using a nine-week spatiotemporal traffic speed data set (road segment × day × time of day) collected in Guangzhou, China, we evaluate the performance of this fully Bayesian model and explore how different data representations affect imputation performance through extensive experiments. The results show the proposed model can produce accurate imputations even under temporally correlated data corruption. Our experiments also show that data representation is a crucial factor for model performance, and a third-order tensor structure outperforms the matrix and fourth-order tensor representations in preserving information in our data set. We hope this work could give insights to practitioners when performing spatiotemporal data imputation tasks.

Effect of missing data on multitask prediction methods

There has been a growing interest in multitask prediction in chemoinformatics, helped by the increasing use of deep neural networks in this field. This technique is applied to multitarget data sets, where compounds have been tested against different targets, with the aim of developing models to predict a profile of biological activities for a given compound. However, multitarget data sets tend to be sparse; i.e., not all compound-target combinations have experimental values. There has been little research on the effect of missing data on the performance of multitask methods. We have used two complete data sets to simulate sparseness by removing data from the training set. Different models to remove the data were compared. These sparse sets were used to train two different multitask methods, deep neural networks and Macau, which is a Bayesian probabilistic matrix factorization technique. Results from both methods were remarkably similar and showed that the performance decrease because of missing data is at first small before accelerating after large amounts of data are removed. This work provides a first approximation to assess how much data is required to produce good performance in multitask prediction exercises.

Personalized query suggestion based on user behavior

Query suggestions help users refine their queries after they input an initial query. Previous work mainly concentrated on similarity-based and context-based query suggestion approaches. However, models that focus on adapting to a specific user (personalization) can help to improve the probability of the user being satisfied. In this paper, we propose a personalized query suggestion model based on users’ search behavior (UB model), where we inject relevance between queries and users’ search behavior into a basic probabilistic model. For the relevance between queries, we consider their semantical similarity and co-occurrence which indicates the behavior information from other users in web search. Regarding the current user’s preference to a query, we combine the user’s short-term and long-term search behavior in a linear fashion and deal with the data sparse problem with Bayesian probabilistic matrix factorization (BPMF). In particular, we also investigate the impact of different personalization strategies (the combination of the user’s short-term and long-term search behavior) on the performance of query suggestion reranking. We quantify the improvement of our proposed UB model against a state-of-the-art baseline using the public AOL query logs and show that it beats the baseline in terms of metrics used in query suggestion reranking. The experimental results show that: (i) for personalized ranking, users’ behavioral information helps to improve query suggestion effectiveness; and (ii) given a query, merging information inferred from the short-term and long-term search behavior of a particular user can result in a better performance than both plain approaches.

Large-Scale Bayesian Probabilistic Matrix Factorization with Memo-Free Distributed Variational Inference

Bayesian Probabilistic Matrix Factorization (BPMF) is a powerful model in many dyadic data prediction problems, especially the applications of Recommender system. However, its poor scalability has limited its wide applications on massive data. Based on the conditional independence property of observed entries in BPMF model, we propose a novel distributed memo-free variational inference method for large-scale matrix factorization problems. Compared with the state-of-the-art methods, the proposed method is favored for several attractive properties. Specifically, it does not require tuning of learning rate carefully, shuffling the training set at each iteration, or storing massive redundant variables, and can introduce new agents into the computations on the fly. We conduct extensive experiments on both synthetic and real-world datasets. The experimental results show that our method can converge significantly faster with better prediction performance than alternative algorithms.

Link sign prediction by Variational Bayesian Probabilistic Matrix Factorization with Student-t Prior

In signed social networks, link sign prediction refers to using the observed link signs to infer the signs of the remaining links, which is important for mining and analyzing the evolution of social networks. The widely used matrix factorization-based approach – Bayesian Probabilistic Matrix Factorization (BMF), assumes that the noise between the real and predicted entry is Gaussian noise, and the prior of latent features is multivariate Gaussian distribution. However, Gaussian noise model is sensitive to outliers and is not robust. Gaussian prior model neglects the differences between latent features, that is, it does not distinguish between important and non-important features. Thus, Gaussian assumption based models perform poorly on real-world (sparse) datasets. To address these issues, a novel Variational Bayesian Probabilistic Matrix Factorization with Student-t prior model (TBMF) is proposed in this paper. A univariate Student-t distribution is used to fit the prediction noise, and a multivariate Student-t distribution is adopted for the prior of latent features. Due to the high kurtosis of Student-t distribution, TBMF can select informative latent features automatically, characterize long-tail cases and obtain reasonable representations on many real-world datasets. Experimental results show that TBMF improves the prediction performance significantly compared with the state-of-the-art algorithms, especially when the observed links are few.

Distributed Bayesian Probabilistic Matrix Factorization

Using the matrix factorization technique in machine learning is very common mainly in areas like recommender systems. Despite its high prediction accuracy and its ability to avoid over-fitting of the data, the Bayesian Probabilistic Matrix Factorization algorithm (BPMF) has not been widely used on large scale data because of the prohibitive cost. In this paper, we propose a distributed high-performance parallel implementation of the BPMF using Gibbs sampling on shared and distributed architectures. We show by using efficient load balancing using work stealing on a single node, and by using asynchronous communication in the distributed version we beat state of the art implementations.

A Method of Bayesian Probabilistic Matrix Factorization Based on Generalized Gaussian Distribution

A Method of Bayesian Probabilistic Matrix Factorization Based on Generalized Gaussian Distribution

Behavior‐based personalization in web search

Personalized search approaches tailor search results to users' current interests, so as to help improve the likelihood of a user finding relevant documents for their query. Previous work on personalized search focuses on using the content of the user's query and of the documents clicked to model the user's preference. In this paper we focus on a different type of signal: We investigate the use of behavioral information for the purpose of search personalization. That is, we consider clicks and dwell time for reranking an initially retrieved list of documents. In particular, we (i) investigate the impact of distributions of users and queries on document reranking; (ii) estimate the relevance of a document for a query at 2 levels, at the query‐level and at the word‐level, to alleviate the problem of sparseness; and (iii) perform an experimental evaluation both for users seen during the training period and for users not seen during training. For the latter, we explore the use of information from similar users who have been seen during the training period. We use the dwell time on clicked documents to estimate a document's relevance to a query, and perform Bayesian probabilistic matrix factorization to generate a relevance distribution of a document over queries. Our experiments show that: (i) for personalized ranking, behavioral information helps to improve retrieval effectiveness; and (ii) given a query, merging information inferred from behavior of a particular user and from behaviors of other users with a user‐dependent adaptive weight outperforms any combination with a fixed weight.

Diversifying Query Auto-Completion

Query auto-completion assists web search users in formulating queries with a few keystrokes, helping them to avoid spelling mistakes and to produce clear query expressions, and so on. Previous work on query auto-completion mainly centers around returning a list of completions to users, aiming to push queries that are most likely intended by the user to the top positions but ignoring the redundancy among the query candidates in the list. Thus, semantically related queries matching the input prefix are often returned together. This may push valuable suggestions out of the list, given that only a limited number of candidates can be shown to the user, which may result in a less than optimal search experience. In this article, we consider the task of diversifying query auto-completion, which aims to return the correct query completions early in a ranked list of candidate completions and at the same time reduce the redundancy among query auto-completion candidates. We develop a greedy query selection approach that predicts query completions based on the current search popularity of candidate completions and on the aspects of previous queries in the same search session. The popularity of completion candidates at query time can be directly aggregated from query logs. However, query aspects are implicitly expressed by previous clicked documents in the search context. To determine the query aspect, we categorize clicked documents of a query using a hierarchy based on the open directory project. Bayesian probabilistic matrix factorization is applied to derive the distribution of queries over all aspects. We quantify the improvement of our greedy query selection model against a state-of-the-art baseline using two large-scale, real-world query logs and show that it beats the baseline in terms of well-known metrics used in query auto-completion and diversification. In addition, we conduct a side-by-side experiment to verify the effectiveness of our proposal.

Bayesian Hierarchical Kernelized Probabilistic Matrix Factorization

We present a Bayesian analysis framework for matrix-variate normal data with dependency structures induced by rows and columns. This framework of matrix normal models includes prior specifications, posterior computation using Markov chain Monte Carlo methods, evaluation of prediction uncertainty, model structure search, and extensions to multidimensional arrays. Compared with Bayesian probabilistic matrix factorization, which integrates a Gaussian prior for single row of the data matrix, our proposed model, namely Bayesian hierarchical kernelized probabilistic matrix factorization, imposes Gaussian Process priors over multiple rows of the matrix. Hence, the learned model explicitly captures the underlying correlation among the rows and the columns. In addition, our method requires no specific assumptions like independence of latent factors for rows and columns, which obtains more flexibility for modeling real data compared to existing works. Finally, the proposed framework can be adapted to a wide range of applications, including multivariate analysis, times series, and spatial modeling. Experiments highlight the superiority of the proposed model in handling model uncertainty and model optimization.

Dynamic Bayesian Probabilistic Matrix Factorization

Collaborative filtering algorithms generally rely on the assumption that user preference patterns remain stationary. However, real-world relational data are seldom stationary. User preference patterns may change over time, giving rise to the requirement of designing collaborative filtering systems capable of detecting and adapting to preference pattern shifts. Motivated by this observation, in this paper we propose a dynamic Bayesian probabilistic matrix factorization model, designed for modeling time-varying distributions. Formulation of our model is based on imposition of a dynamic hierarchical Dirichlet process (dHDP) prior over the space of probabilistic matrix factorization models to capture the time-evolving statistical properties of modeled sequential relational datasets. We develop a simple Markov Chain Monte Carlo sampler to perform inference. We present experimental results to demonstrate the superiority of our temporal model.

Bayesian Probabilistic Matrix Factorization with Social Relations and Item Contents for recommendation

Recommendation systems have received great attention for their commercial value in today's online business world. However, most recommendation systems encounter the data sparsity problem and the cold-start problem. To improve recommendation accuracy in this circumstance, additional sources of information about the users and items should be incorporated in recommendation systems. In this paper, we modify the model in Bayesian Probabilistic Matrix Factorization, and propose two recommendation approaches fusing social relations and item contents with user ratings in a novel way. The proposed approach is computationally efficient and can be applied to trust-aware or content-aware recommendation systems with very large dataset. Experimental results on three real world datasets show that our method gets more accurate recommendation results with faster converging speed than other matrix factorization based methods. We also verify our method in cold-start settings, and our method gets more accurate recommendation results than the compared approaches.