User-item Pair Research Articles

A novel deep learning-based recommender system for Arabic content

In the era of big data, recommender systems (RSs) have become growing essential tools. They represent important machine learning solutions that mainly contribute to keeping users engaged with personalized content in e-platforms. Several RSs have been proposed in the literature, and most of them have focused on English content. However, for content in other languages like Arabic, very restricted works have been done to develop RSs. In recent times, the Arabic content on the Web has increased significantly because of the growing number of Arabic web users. This highlights the need for building RSs for Arabic content. To better handle this challenge, we decided to provide the research community with a novel deep learning (DL)-based RS devoted to Arabic content. The main goal of the proposed RS is to predict user preferences from textual reviews written in the Arabic language. This is achieved by combining two independent DL techniques into one system: a convolutional neural network (CNN)-text processor for representing users and items; and a neural network, in particular, a multi-layer perceptron (MLP) to estimate interactions between user-item pairs. Extensive experiments on four large-scale Arabic datasets demonstrate that our proposed system can achieve better prediction accuracy than other state-of-the-art alternatives. Notably, it improves the MSE between 0.84% and 16.96%, and the MAE between 0.14% and 13.71%. This work is the first attempt designed to deal with a large volume of data in the Arabic context, opening up new research possibilities for future developments of Arabic RSs.

Knowledge graph-based multi-context-aware recommendation algorithm

Because a knowledge graph’s huge amount of item information can help recommender systems develop user/item representations, it has become the most important source of side information. Regardless of the numerous types of user/item representation approaches used in knowledge graph-based recommendation scenarios, they all have problems. In this paper, we propose a knowledge graph-based multi-context-aware recommendation algorithm for learning user/item representations that combines the advantages of path-based and propagation-based methods. A new concept (i.e., rule) is proposed first, which can be a useful way to characterize the user’s preferences. Next, based on user-item interactions, an automatic rule discovery algorithm is proposed that can automatically select the most representative user preferences templates in a given recommendation scenario based on the knowledge graph and user behaviors. Then, the learning of high-order connectivity between long-distance user-item pairs is realized according to these templates. After that, a feature representation method of the local neighborhood characteristics of users and items is introduced to compensate for the defect that the path-based method can only catch the high-order connectivity. The experimental results demonstrate MANN’s superiority over eight state-of-the-art baselines.

CoCNN: Co-occurrence CNN for recommendation

Under the assumption that items are independent and identically distributed, most existing CF methods learn representation from user–item pairs, but ignore the connections among items, leading to limited performance. Considering the challenge of recommendation, we propose a novel neural network, CoCNN, which combines a Co-occurrence pattern and CNN for CF with implicit feedback. The key idea of the co-occurrence pattern is that some items always appear between pairs on a user’s favorite list. In CoCNN, co-occurrence relationships act as a bridge in user–item pairs and item–item pairs, which are not observed directly. To model user–item and item–item information simultaneously, we propose a multi-task neural network to share the knowledge of the two tasks. Finally, experimental results demonstrate that CoCNN successfully captures more useful information, and therefore can be used as a simple and effective tool for recommendation. Our projects are available online at https://github.com/XiuzeZhou/CoCNN.

Accurately Estimating User Cardinalities and Detecting Super Spreaders Over Time

Online monitoring user cardinalities in graph streams is fundamental for many applications such as anomaly detection. These graph streams may contain edge duplicates and have a large number of user-item pairs, which makes it infeasible to exactly compute user cardinalities due to limited computational and memory resources. Existing methods are designed to approximately estimate user cardinalities, but their accuracy highly depends on complex parameters and they cannot provide anytime-available estimation. To address these problems, we develop novel bit/register sharing algorithms, which use a bit/register array to build a compact sketch of all users’ connected items. Our algorithms exploit the dynamic properties of the bit/register arrays (e.g., the fraction of zero bits in the bit array) to significantly improve the estimation accuracy, and have low time complexity <inline-formula><tex-math notation="LaTeX">$O(1)$</tex-math></inline-formula> to update the estimations for a new user-item pair. In addition, our algorithms are simple and easy to use, without requirements to tune any parameter. Furthermore, we extend our methods to detect super spreaders with large cardinalities in real-time. We evaluate the performance of our methods on real-world datasets. The experimental results demonstrate that our methods are several times more accurate and faster than state-of-the-art methods using the same amount of memory.

Attention Over Self-Attention: Intention-Aware Re-Ranking With Dynamic Transformer Encoders for Recommendation

Re-ranking models refine the item recommendation list generated by the prior global ranking model with intra-item relationships. However, most existing re-ranking solutions refine recommendation list based on the implicit feedback with a shared re-ranking model, which regrettably ignore the intra-item relationships under diverse user intentions. In this paper, we propose a novel Intention-aware Re-ranking Model with Dynamic Transformer Encoder (RAISE), aiming to perform user-specific prediction for each target user based on her intentions. Specifically, we first propose to mine latent user intentions from text reviews with an intention discovering module (IDM). By differentiating the importance of review information with a co-attention network, the latent user intention can be explicitly modeled for each user-item pair. We then introduce a dynamic transformer encoder (DTE) to capture user-specific intra-item relationships among item candidates by seamlessly accommodating the learnt latent user intentions via IDM. As such, RAISE is able to perform user-specific prediction without increasing the depth (number of blocks) and width (number of heads) of the prediction model. Empirical study on four public datasets shows the superiority of our proposed RAISE, with up to 13.95%, 12.30%, and 13.03% relative improvements evaluated by Precision, MAP, and NDCG respectively.

NEAR: Non-supervised Explainability Architecture for Accurate Review-Based Collaborative Filtering

There is a critical issue in explainable recommender systems that compounds the challenges of explainability yet is rarely tackled: the lack of ground-truth explanation texts for training. It is unrealistic to expect every user-item pair in a dataset to have a corresponding target explanation. Hence, we pioneer the first non-supervised explainability architecture for review-based collaborative filtering (called NEAR) as our novel contribution to the theory of explanation construction in recommender systems. While maintaining excellent recommendation performance, our approach reformulates explainability as a non-supervised (i.e., unsupervised and self-supervised) explanation generation task. We formally define two explanation types, both of which NEAR can produce. An invariant explanation, fixed for all users, is based on the unsupervised extractive summary of an item's reviews via embedding clustering. Meanwhile, a variant explanation, personalized for a specific user, is a sentence-level text generated by our customized Transformer conditioned on every user-item-rating tuple and artificial ground-truth (self-supervised label) from one of the invariant explanation's sentences. Our empirical evaluation illustrates that NEAR's rating prediction accuracy is better than the other state-of-the-art baselines. Moreover, experiments and assessments show that NEAR-generated variant explanations are more personalized and distinct than those from other Transformer-based models, and our invariant explanations are preferred over those from other contemporary models in real life.

Region or Global A Principle for Negative Sampling in Graph-based Recommendation

Graph-based recommendation systems are blossoming recently, which models user-item interactions as a user-item graph and utilizes graph neural networks (GNNs) to learn the embeddings for users and items. A fundamental challenge of graph-based recommendation is that there only exists observed positive user-item pairs in the user-item graph. Negative sampling is a vital technique to solve the one-class problem and is widely used in many recommendation methods. However, the previous works only focus on the design of negative sampling distribution but ignore the sampled region for negative sampling. In this work, we propose the Three-Region Principle to guide negative sampling, which suggests that we should negatively sample more items at an intermediate region and less adjacent and distant items. In light of this principle, we present the RecNS method, which is a general negative sampling method designed with two sampling strategies: positive-assisted sampling and exposure-augmented sampling. Instead of sampling existing negative items from graph data, we merge these two strategies in embedding space to generate negative item embeddings. Extensive experiments demonstrate that our proposed RecNS method significantly outperforms all negative sampling baselines, e.g., 10.47% for PinSage, 6.02% for NGCF, and 8.20% for LightGCN in terms of Recall@20 on the Alibaba dataset.

High-order autoencoder with data augmentation for collaborative filtering

Early DNN-based collaborative filtering (CF) approaches have demonstrated their superior performance than traditional CF such as Matrix Factorization. However, such approaches treat each user–item interaction as separate data and thus overlook the intrinsic relationships among data instances. Inspired by the discovery that the autoencoder architecture can force the hidden representation to capture information about the structure of the graph data, in this work, we propose a novel framework called High-order Autoencoder based Collaborative Filtering (HACF) that enhances the classic NeuMF framework with autoencoders for capturing latent high-order connectivity signals in the user–item interaction graph. Specifically, each user–item pair is augmented with higher-order neighbours and input to two sets of autoencoders, one set for the users and the other for the items. All the autoencoders in one set share parameters so increasing the number of autoencoders does not increase the model size.We have conducted extensive experiments on four popular public benchmark datasets with different sparsity. The overall comparison results demonstrate the advantages of autoencoder-based methods and show that our framework outperforms some state-of-the-art DNN-based collaborative filtering approaches.

HyperSoRec: Exploiting Hyperbolic User and Item Representations with Multiple Aspects for Social-aware Recommendation

Social recommendation has achieved great success in many domains including e-commerce and location-based social networks. Existing methods usually explore the user-item interactions or user-user connections to predict users’ preference behaviors. However, they usually learn both user and item representations in Euclidean space, which has large limitations for exploring the latent hierarchical property in the data. In this article, we study a novel problem of hyperbolic social recommendation, where we aim to learn the compact but strong representations for both users and items. Meanwhile, this work also addresses two critical domain-issues, which are under-explored. First, users often make trade-offs with multiple underlying aspect factors to make decisions during their interactions with items. Second, users generally build connections with others in terms of different aspects, which produces different influences with aspects in social network. To this end, we propose a novel graph neural network (GNN) framework with multiple aspect learning, namely, HyperSoRec. Specifically, we first embed all users, items, and aspects into hyperbolic space with superior representations to ensure their hierarchical properties. Then, we adapt a GNN with novel multi-aspect message-passing-receiving mechanism to capture different influences among users. Next, to characterize the multi-aspect interactions of users on items, we propose an adaptive hyperbolic metric learning method by introducing learnable interactive relations among different aspects. Finally, we utilize the hyperbolic translational distance to measure the plausibility in each user-item pair for recommendation. Experimental results on two public datasets clearly demonstrate that our HyperSoRec not only achieves significant improvement for recommendation performance but also shows better representation ability in hyperbolic space with strong robustness and reliability.

GFE: General Knowledge Enhanced Framework for Explainable Sequential Recommendation

It is vital for sequential recommendation to provide accurate and explainable results for user, which can help them make better decisions. In this paper, we develop a General Knowledge Enhanced Framework for Explainable Sequential Recommendation (GFE) to capture user’s fine-grained preferences and dynamic preferences evolution. Specifically, the fine-grained preferences are modeled as intrinsic interests and external potential interests, which can be captured by sequential-aware interest and knowledge-aware interest modules respectively. Moreover, the high-order paths between each user-item pair are generated with the help of the knowledge graph, which contain abundant high-order semantic relevance among entities. To make better use of this character, we propose a hierarchical self-attention mechanism to aggregate the high-order semantic information from these knowledge paths, thus discovering user’s dynamic preferences evolution. According to these abilities, GFE can provide more reasonable explanations from the views of microcosm and macrocosm. Unlike other traditional explainable sequential recommendation models, GFE has the strong generalization to integrate with other pure sequential recommendation models and endow explainability to them. Based on this nature, we combine the GFE with two state-of-the-art sequential recommendation models and further propose two GFE-based models, called GFE-SASRec and GFE-TiSASRec, to show the availability of GFE. Finally, experiments conducted on three real-world public datasets demonstrate the state-of-the-art performance and the strong explainability of GFE.

Constrained Dual-Level Bandit for Personalized Impression Regulation in Online Ranking Systems

Impression regulation plays an important role in various online ranking systems, e.g. , e-commerce ranking systems always need to achieve local commercial demands on some pre-labeled target items like fresh item cultivation and fraudulent item counteracting while maximizing its global revenue. However, local impression regulation may cause “butterfly effects” on the global scale, e.g. , in e-commerce, the price preference fluctuation in initial conditions (overpriced or underpriced items) may create a significantly different outcome, thus affecting shopping experience and bringing economic losses to platforms. To prevent “butterfly effects”, some researchers define their regulation objectives with global constraints, by using contextual bandit at the page-level that requires all items on one page sharing the same regulation action, which fails to conduct impression regulation on individual items. To address this problem, in this article, we propose a personalized impression regulation method that can directly makes regulation decisions for each user-item pair. Specifically, we model the regulation problem as a C onstrained D ual-level B andit (CDB) problem, where the local regulation action and reward signals are at the item-level while the global effect constraint on the platform impression can be calculated at the page-level only. To handle the asynchronous signals, we first expand the page-level constraint to the item-level and then derive the policy updating as a second-order cone optimization problem. Our CDB approaches the optimal policy by iteratively solving the optimization problem. Experiments are performed on both offline and online datasets, and the results, theoretically and empirically, demonstrate CDB outperforms state-of-the-art algorithms.

Hierarchical attentive knowledge graph embedding for personalized recommendation

Knowledge graphs (KGs) have proven to be effective for high-quality recommendation, where the connectivities between users and items provide rich and complementary information to user-item interactions. Most existing methods, however, are insufficient to exploit the KGs for capturing user preferences, as they either represent the user-item connectivities via paths with limited expressiveness or implicitly model them by propagating information over the entire KG with inevitable noise. In this paper, we design a novel hierarchical attentive knowledge graph embedding (HAKG) framework to exploit the KGs for effective recommendation. Specifically, HAKG first extracts the expressive subgraphs that link user-item pairs to characterize their connectivities, which accommodate both the semantics and topology of KGs. The subgraphs are then encoded via a hierarchical attentive subgraph encoding to generate effective subgraph embeddings for enhanced user preference prediction. Extensive experiments show the superiority of HAKG against state-of-the-art recommendation methods, as well as its potential in alleviating the data sparsity issue.

CoNet: Co-occurrence neural networks for recommendation

Assuming that both users and items are independent and identically distributed, most existing methods model user–item pairs, while ignoring the relationship between items, leading to limited performance. To solve this problem, we propose a novel neural network, CoNet, which can effectively model the co-occurrence pattern for Collaborative Filtering (CF). We argue that items always occur in pairs, i.e. an item co-occurrence pattern. For example, movies ”Harry Potter 1” and ”Harry Potter 2” are always viewed by users who like magic style films. To learn the latent features, CoNet is simultaneously modeled on user–item and item–item interactions. Compared with methods that train on a single user–item pair, CoNet can encode highly descriptive features from the co-occurrence pattern.To achieve a better performance, we design an attention network to learn the weight of a user’s preference for different items and subsequently aggregate the weighted embeddings to obtain the co-occurrence representations. Finally, we conducted extensive experiments using several data sets, which show that the proposed method is superior to other baseline approaches. Source code of CoNet is available from https://github.com/XiuzeZhou/conet.

NeuSE: A Neural Snapshot Ensemble Method for Collaborative Filtering

In collaborative filtering (CF) algorithms, the optimal models are usually learned by globally minimizing the empirical risks averaged over all the observed data. However, the global models are often obtained via a performance tradeoff among users/items, i.e., not all users/items are perfectly fitted by the global models due to the hard non-convex optimization problems in CF algorithms. Ensemble learning can address this issue by learning multiple diverse models but usually suffer from efficiency issue on large datasets or complex algorithms. In this article, we keep the intermediate models obtained during global model learning as the snapshot models, and then adaptively combine the snapshot models for individual user-item pairs using a memory network-based method. Empirical studies on three real-world datasets show that the proposed method can extensively and significantly improve the accuracy (up to 15.9% relatively) when applied to a variety of existing collaborative filtering methods.

Towards purchase prediction: A transaction-based setting and a graph-based method leveraging price information

Targeting at boosting business revenue, purchase prediction based on user behavior is crucial to e-commerce. However, it is not a well-explored topic due to a lack of relevant datasets. Specifically, no public dataset provides both price and discount information varying on time, which play an essential role in the user’s decision making. Besides, existing learn-to-rank methods cannot explicitly predict the purchase possibility for a specific user-item pair. In this paper, we propose a two-step graph-based model, where the graph model is applied in the first step to learn representations of both users and items over click-through data, and the second step is a classifier incorporating the price information of each transaction record. To evaluate the model performance, we propose a transaction-based framework focusing on the purchased items and their context clicks, which contain items that a user is interested in but fails to choose after comparison. Our experiments show that exploiting the price and discount information can significantly enhance prediction accuracy.

Multi-level attentive deep user-item representation learning for recommendation system

With the development of e-commerce platforms, user reviews have become a vital source of information to address the sparsity problems for enhancing the predictive performance of the recommendation systems (RSs). However, the traditional methods of the RSs used to model user/item latent features based on static vectors in an independent manner without considering the dynamic nature of the user-item interactions which potentially affect the accuracy of the recommendation process. Thus, this paper proposes a RS model that exploits neural attention techniques to learn user/item representations by jointly considering the fine-grained semantic information for the user-item pairs. The proposed model utilizes both review-based and interaction-specific features for the user/item reviews to learn heterogeneous user/item representations. First, a BiLSTM sequence encoder is used to learn the contextual information of words, and a Co-attention network is then designed to jointly capture the most relevant semantic information of reviews for the user-item pair. To better capture user/item latent factors comprehensively, interaction-specific features based on the rating scores are further integrated with the review-specific latent features via a shared hidden layer. Finally, an attentive factorization machine (FM) is then applied on the shared hidden layer of the integrated user/item features for the final prediction. We carry out a series of experiments using real-world datasets and the results demonstrate that our proposed method is better than the baseline approaches in terms of both rating prediction and ranking performance.

Combining user-end and item-end knowledge graph learning for personalized recommendation

How to accurately model user preferences based on historical user behaviour and auxiliary information is of great importance in personalized recommendation tasks. Among all types of auxiliary information, knowledge graphs (KGs) are an emerging type of auxiliary information with nodes and edges that contain rich structural information and semantic information. Many studies prove that incorporating KG into personalized recommendation tasks can effectively improve the performance, rationality and interpretability of recommendations. However, existing methods either explore the independent meta-paths for user-item pairs in KGs or use a graph convolution network on all KGs to obtain embeddings for users and items separately. Although both types of methods have respective effects, the former cannot fully capture the structural information of user-item pairs in KGs, while the latter ignores the mutual effect between the target user and item during the embedding learning process. To alleviate the shortcomings of these methods, we design a graph convolution-based recommendation model called Combining User-end and Item-end Knowledge Graph Learning (CUIKG), which aims to capture the relevance between users’ personalized preferences and items by jointly mining the associated attribute information in their respective KG. Specifically, we describe user embedding from a user KG and then introduce user embedding, which contains the user profile into the item KG, to describe item embedding with the method of Graph Convolution Network. Finally, we predict user preference probability for a given item via multilayer perception. CUIKG describes the connection between user-end KG and item-end KG, and mines the structural and semantic information present in KG. Experimental results with two real-world datasets demonstrate the superiority of the proposed method over existing methods.

Neural Collaborative Preference Learning With Pairwise Comparisons

Collaborative Ranking (CR), as an effective recommendation framework, has attracted increasing attention in recent years. Most CR methods simply adopt the inner product between user/item embeddings as the rating score function, with an assumption that the interacted items are preferred to non-interacted ones. However, such fixed score functions and assumption might not be sufficient to capture the real preference ranking list from the complicated interactions in real-world data. To alleviate this issue, we develop a novel collaborative ranking framework that learns an arbitrary utility function for item ranking with user preference concerned. In the core of our framework, a neural network is employed to model the utility function for personalized ranking with the strength of its nonlinearity. On top of this, we further adopt a pairwise ranking loss for user-item pairs to preserve the preference order of items for users. Besides, such utility function enables us to generate the final top-K preference list in a much easier way. Finally, extensive experiments on four real-world datasets show the validity of our proposed method.

Deep neural review text interaction for recommendation systems

Users’ reviews contain valuable information which are not taken into account in most recommender systems. According to the latest studies in this field, using review texts could not only improve the performance of recommendation, but it can also alleviate the impact of data sparsity and help to tackle this problem. In this paper, we present a neural recommender model which recommends items by leveraging user reviews. In order to predict user rating for each item, our proposed model, named MatchPyramidRecommenderSystem (MPRS), represents each user and item with their corresponding review texts. Thus, the problem of recommendation is viewed as a text matching problem such that the matching score obtained from matching user and item texts could be considered as a good representative of their joint extent of similarity. To solve the text matching problem, inspired by MatchPyramid (Pang et al., 2016), we employed an interaction-based approach according to which a matching matrix is constructed given a pair of input texts. The matching matrix, which has the property of hierarchical matching patterns, is then fed into a Convolutional Neural Network (CNN) to compute the matching score for the given user–item pair. Our experiments on the small data categories of Amazon review dataset show that our proposed model gains from 1.76% to 21.72% relative improvement compared to DeepCoNN model (Zheng et al., 2017), and from 0.83% to 3.15% relative improvement compared to TransNets model (Catherine and Cohen, 2017). Also, on two large categories, namely AZ-CSJ and AZ-Mov, our model achieves relative improvements of 8.08% and 7.56% compared to the DeepCoNN model, and relative improvements of 1.74% and 0.86% compared to the TransNets model, respectively.

A Deep Hybrid Model for Recommendation by jointly leveraging ratings, reviews and metadata information

Although matrix factorization (MF) based collaborative filtering (CF) and deep learning approaches have achieved great success, there is still much room for improvement in recommender systems. Most of the existing approaches mainly adopt product ratings, reviews or content features in order to predict unknown rating for a user–item pair. In the discourse matter, some recent works attempted to obtain better latent representations of users and items by integrating different multi-source data, however, the heterogeneity of data is still a problem deserving study. Such models usually face two issues: (1) They extract the representations in a static and independent manner, thus ignoring the correlations between latent features learned from different information sources. (2) There is no unified framework that can mutually learn latent features from different sources such as ratings, reviews and meta-data of users, items and reviews. In the proposed model, called A Deep Hybrid Model for Recommendation (DHMR), we propose a joint deep model for learning higher-order non-linear latent feature interactions from reviews and metadata information. Further, we incorporate user–item interactions (from user–item ratings matrix) adopting MF model into the neural network. Thus, the proposed model consists of two parallel neural networks and an MF based model that are integrated by the attention and MLP layers at the top, learning lower-order (linear and non-linear) feature interactions of users and items separately and higher-order non-linear feature interactions jointly. Extensive experiments on real-world datasets demonstrate that DHMR significantly outperforms state-of-the-art recommendation models.