Modelling socially-influenced conditional preferences over feature values in recommender systems based on factorised collaborative filtering

How to accurately represent a high-dimensional and sparse (HiDS) user-item rating matrix is a crucial issue in implementing a recommender system. A latent factor (LF) model is one of the most popular and successful approaches to address this issue. It is developed by minimizing the errors between the observed entries and the estimated ones on an HiDS matrix. Current studies commonly employ L 2 -norm to minimize the errors because it has a smooth gradient, making a resultant LF model can accurately represent an HiDS matrix. As is well known, however, L 2 -norm is very sensitive to the outlier data or called unreliable ratings in the context of the recommender system. Unfortunately, the unreliable ratings often exist in an HiDS matrix due to some malicious users. To address this issue, this paper proposes a Smooth L 1 -norm-oriented Latent Factor (SL 1 -LF) model. Its main idea is to employ smooth L 1 -norm rather than L 2 -norm to minimize the errors, making it have both high robustness and accuracy in representing an HiDS matrix. Experimental results on four HiDS matrices generated by industrial recommender systems demonstrate that the proposed SL 1 -LF model is robust to the outlier data and has significantly higher prediction accuracy than state-of-the-art models for the missing data of an HiDS matrix.

High-dimensional and sparse (HiDS) matrices generated by recommender systems (RSs) contain rich knowledge. A latent factor (LF) model can address such data effectively. Stochastic gradient descent (SGD) is an efficient algorithm for building a LF model on an HiDS matrix. However, it suffers slow convergence. To address this issue, this study proposes to implement a LF model with a proportional integral derivative (PID) controller. The main idea is to continuously apply a correction for SGD to accelerate the training process. Based on such design, a PID-based LF (PLF) model is proposed. Empirical studies on two HiDS matrices from RSs indicate that a PLF model outperforms an LF model in terms of both convergence rate and prediction accuracy for missing data.

A latent factor (LF) model can implement efficient analysis for a high-dimensional and sparse (HiDS) matrix from recommender systems (RSs). However, an LF model's representation learning ability to a targeted HiDS matrix is heavily proportional to its known data density. Unfortunately, an HiDS matrix's known data are limited due to users' activity limitations in RSs. Motivated by this observation, this paper proposes a Prediction-sampling-based Multilayer-structured Latent Factor (PMLF) model. Following the principle of Deep Forest [1], PMLF implements a loosely-connected multilayered LF structure, where each layer generates synthetic ratings to enrich the input for the next layer. Such an injection process is carefully monitored through a random sampling process and nonlinear activations to avoid overfitting. Thus, PMLF's representation learning ability to an HiDS matrix is significantly enhanced owing to the carefully injected estimates and its generalized multilayer-structure. Experimental results on four HiDS matrices from industrial RSs indicate that compared with six state-of-the-art LF-based and deep neural networks-based models, PMLF well balances the prediction accuracy and computational efficiency, making it satisfy demands of fast and accurate industrial applications.

Performing highly accurate representation learning on a high-dimensional and sparse (HiDS) matrix is of great significance in a big data-related application such as a recommender system. A latent factor (LF) model is one of the most efficient approaches to the HiDS matrix representation. However, an LF model's representation learning ability relies heavily on an HiDS matrix's known data density, which is extremely low due to numerous missing data entities. To address this issue, this work proposes a prediction-sampling-based multilayer-structured LF (PMLF) model with twofold ideas: 1) constructing a loosely connected multilayered LF architecture to increase the known data density of an input HiDS matrix by generating synthetic data layer by layer and 2) constraining this synthetic data generating process through a random prediction-sampling strategy and nonlinear activations to avoid overfitting. In the experiments, PMLF is compared with six state-of-the-art LF-and deep neural network (DNN)-based models on four HiDS matrices from industrial applications. The results demonstrate that PMLF outperforms its peers in well-balancing prediction accuracy and computational efficiency.

Performance of latent factor models with extended linear biases

In the area of electronic commerce, recommender systems have become more and more popular. The quality of recommendations depends on the quality of the preference model extracted by the recommender system. Recently, latent factor models based on probabilistic matrix factorisation have gained great attention in both industry and academia, owing to their superior accuracy over traditional recommender systems. Although latent factor models are very efficient, the latency of the features captured in these models impedes explaining the learnt model to the users. A lack of understanding of the latent features makes it difficult to decide on the optimal number of features to give as input to these models. Therefore, the model accuracy degrades when less relevant features are introduced into the model. To tackle this problem, in this paper we propose an extension to the basic matrix factorisation, so that the model takes into account the relevancy of the features beside their values. We test the accuracy of the proposed method on two benchmark datasets. The experiments show that the proposed method makes remarkable improvements over the basic method and some of the state of the art latent factor models.

Recommender system (RS) commonly describes its user-item preferences with a high-dimensional and sparse (HiDS) matrix. A latent factor (LF) model relying on stochastic gradient descent (SGD) is frequently adopted to extract useful information from such an HiDS matrix. In spite of its efficiency, an SGD-based LF model commonly takes many iterations to converge. When processing a large-scale HiDS matrix, its computational efficiency should be further improved by further accelerating its convergence rate as well as maintaining its learning ability. To address this issue, this paper innovatively proposes novel SGD algorithm which incorporates a nonlinear proportional integral derivative (NPID) controller into its learning scheme for building an LF model. The main idea is to adopt an NPID controller to model the learning residual achieved in the past iterations, thereby adjusting the learning direction and step size of the current iteration, thereby making a resultant model converge fast. With the NPID-incorporated SGD algorithm, this study proposes an NPID-SGD-based LF (NSLF) model. Experimental results on two HiDS matrices demonstrate that compared with a standard SGD-based LF model, the proposed model achieves higher computational efficiency and prediction accuracy for missing data of an HiDS matrix.

High-dimensional and incomplete (HDI) data are frequently encountered in big date-related applications for describing restricted observed interactions among large node sets. How to perform accurate and efficient representation learning on such HDI data is a hot yet thorny issue. A latent factor (LF) model has proven to be efficient in addressing it. However, the objective function of an LF model is nonconvex. Commonly adopted first-order methods cannot approach its second-order stationary point, thereby resulting in accuracy loss. On the other hand, traditional second-order methods are impractical for LF models since they suffer from high computational costs due to the required operations on the objective's huge Hessian matrix. In order to address this issue, this study proposes a generalized Nesterov-accelerated second-order LF (GNSLF) model that integrates twofold conceptions: 1) acquiring proper second-order step efficiently by adopting a Hessian-vector algorithm and 2) embedding the second-order step into a generalized Nesterov's acceleration (GNA) method for speeding up its linear search process. The analysis focuses on the local convergence for GNSLF's nonconvex cost function instead of the global convergence has been taken; its local convergence properties have been provided with theoretical proofs. Experimental results on six HDI data cases demonstrate that GNSLF performs better than state-of-the-art LF models in accuracy for missing data estimation with high efficiency, i.e., a second-order model can be accelerated by incorporating GNA without accuracy loss.

Recommender systems (RSs) commonly adopt a user-item rating matrix to describe users' preferences on items. With users and items exploding, such a matrix is usually high-dimensional and sparse (HiDS). Recently, the idea of deep learning has been applied to RSs. However, current deep-structured RSs suffer from high computational complexity. Enlightened by the idea of deep forest, this paper proposes a deep latent factor model (DLFM) for building a deep-structured RS on an HiDS matrix efficiently. Its main idea is to construct a deep-structured model by sequentially connecting multiple latent factor (LF) models instead of multilayered neural networks through a nonlinear activation function. Thus, the computational complexity grows linearly with its layer count, which is easy to resolve in practice. The experimental results on four HiDS matrices from industrial RSs demonstrate that when compared with state-of-the-art LF models and deep-structured RSs, DLFM can well balance the prediction accuracy and computational efficiency, which well fits the desire of industrial RSs for fast and right recommendations.

Latent factor (LF) models are highly effective in extracting useful knowledge from High-Dimensional and Sparse (HiDS) matrices which are commonly seen in various industrial applications. An LF model usually adopts iterative optimizers, which may consume many iterations to achieve a local optima, resulting in considerable time cost. Hence, how to accelerate the training process of an LF model becomes a highly significant issue. To address it, this work innovatively proposes a randomized latent factor (RLF) model. It incorporates the principle of randomized learning techniques for neural networks into the LF analysis on HiDS matrices to alleviate the computational burden greatly. It also extends the standard learning process for randomized neural networks in context of LF analysis to make the resulting model represent an HiDS matrix correctly. Experimental results on three HiDS matrices from industrial applications demonstrate that compared with state-of-the-art LF models, RLF is able to achieve significantly higher computational efficiency and comparable prediction accuracy for missing data. More importantly, it provides a novel, effective, and efficient approach to LF analysis on HiDS matrices.

Latent Factor Models (LFMs) based on Collaborative Filtering (CF) have been widely applied in many recommendation systems, due to their good performance of prediction accuracy. In addition to users' ratings, auxiliary information such as item features is often used to improve performance, especially when ratings are very sparse. To the best of our knowledge, most existing LFMs integrate different item features in the same way for all users. Nevertheless, the attention on different item attributes varies a lot from user to user. For personalized recommendation, it is valuable to know what feature of an item a user cares most about. Besides, the latent vectors used to represent users or items in LFMs have few explicit meanings, which makes it difficult to explain why an item is recommended to a specific user. In this work, we propose the Attention-driven Factor Model (AFM), which can not only integrate item features driven by users' attention but also help answer this why. To estimate users' attention distributions on different item features, we propose the Gated Attention Units (GAUs) for AFM. The GAUs make it possible to let the latent factors talk, by generating user attention distributions from user latent vectors. With users' attention distributions, we can tune the weights of item features for different users. Moreover, users' attention distributions can also serve as explanations for our recommendations. Experiments on several real-world datasets demonstrate the advantages of AFM (using GAUs) over competitive baseline algorithms on rating prediction.

The valuable knowledge contained in High-dimensional and Sparse (HiDS) matrices can be efficiently extracted by a latent factor (LF) model. Regularization techniques are widely incorporated into an LF model to avoid overfitting. The regularization coefficient is very crucial to the prediction accuracy of models. However, its tuning process is time-consuming and boring. This study aims at making the regularization coefficient of a regularized LF model self-adaptive. To do so, an adaptive particle swarm optimization (APSO) algorithm is introduced into a regularized LF model to automatically select the optimal regularization coefficient. Then, to enhance the global search capability of particles, we further propose an APSO and particle swarm optimization (PSO)-incorporated (AP) algorithm, thereby achieving an AP-based LF (APLF) model. Experimental results on four HiDS matrices generated by real applications demonstrate that an APLF model can achieve an automatic selection of regularization coefficient, and is superior to a regularized LF model in terms of prediction accuracy and computational efficiency.

Latent factor (LF)-based models have been proven to be efficient in implementing recommender systems, owing to their ability to well represent high-dimensional and sparse matrices. While prior works focus on boosting both the prediction accuracy and computation efficiency of original LF model by adding linear biases to it, the individual and combinational effects by linear biases in such performance gain remains unclear. To address this issue, this work thoroughly investigates the effect of prior linear biases and training linear biases. We have investigated the parameter update rules and training processes of an LF model with different combinations of linear biases. Empirical validations are conducted on a high dimensional and sparse matrix from industrial systems currently in use. The results show that each linear bias does have positive/negative effects in the performance of an LF model. Such effects are partially data dependent; however, some linear biases like the global average can bring stable performance gain into an LF model. The theoretical and empirical results along with analysis provide guidance in designing the bias scheme in an LF model for recommender systems.

Joint latent factors and attributes to discover interpretable preferences in recommendation

Latent factor (LF) models are greatly efficient in extracting valuable knowledge from High-Dimensional and Sparse (HiDS) matrices which are commonly seen in many industrial applications. Stochastic gradient descent (SGD) is an efficient scheme to build an LF model, yet its convergence rate depends vastly on the learning rate which should be tuned with care. Therefore, automatic selection of an optimal learning rate for an SGD-based LF model is a significant issue. To address it, this study incorporates the principle of particle swarm optimization (PSO) into an SGD-based LF model for searching an optimal learning rate automatically. With it, we further propose an adaptive Latent Factor (ALF) model. Empirical studies on two HiDS matrices from industrial applications indicate that an ALF model obviously outperforms an LF model in terms of convergence rate, and maintains competitive prediction accuracy for missing data.