User-item Pair Research Articles

Matrix Factorization and Prediction for High-Dimensional Co-Occurrence Count Data via Shared Parameter Alternating Zero Inflated Gamma Model

High-dimensional sparse matrix data frequently arise in various applications. A notable example is the weighted word–word co-occurrence count data, which summarizes the weighted frequency of word pairs appearing within the same context window. This type of data typically contains highly skewed non-negative values with an abundance of zeros. Another example is the co-occurrence of item–item or user–item pairs in e-commerce, which also generates high-dimensional data. The objective is to utilize these data to predict the relevance between items or users. In this paper, we assume that items or users can be represented by unknown dense vectors. The model treats the co-occurrence counts as arising from zero-inflated Gamma random variables and employs cosine similarity between the unknown vectors to summarize item–item relevance. The unknown values are estimated using the shared parameter alternating zero-inflated Gamma regression models (SA-ZIG). Both canonical link and log link models are considered. Two parameter updating schemes are proposed, along with an algorithm to estimate the unknown parameters. Convergence analysis is presented analytically. Numerical studies demonstrate that the SA-ZIG using Fisher scoring without learning rate adjustment may fail to find the maximum likelihood estimate. However, the SA-ZIG with learning rate adjustment performs satisfactorily in our simulation studies.

Collaborative Meta-Path Modeling for Explainable Recommendation

Although recommender systems have achieved considerable success, sometimes it is difficult to convince users due to the failure to explain the recommendation results. For this reason, explainable recommender systems have drawn a lot of attention in recent years. Among explainable recommendation models, the meta-path-based model plays a significant role because it can reason over the path connecting a user–item pair to achieve explainability. However, it is difficult for the meta-path-based model to achieve such a common explanation in collaborative filtering as “a user similar to you has purchased item <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$A$</tex-math> </inline-formula> ” because there is no such meta-path. In this article, we contribute a new model named collaborative meta-path modeling for explainable recommendation (COMPER). It models the similarity of user pairs and item pairs through rating information and constructs collaborative meta-paths for explainability. In addition, we design an attention mechanism to aggregate different paths connecting the target user and the target item. Moreover, the information of the subgraph composed of all paths connecting the target user and the target item is integrated for rating prediction. Extensive experiments on five real-world datasets demonstrate that COMPER achieves good performance in a variety of scenarios, achieving improvements over several baselines.

Mitigating Confounding Bias in Practical Recommender Systems With Partially Inaccessible Exposure Status.

To improve user experience, recommender systems have been widely used on many online platforms. In these systems, recommendation models are typically learned from positive/negative feedback that are collected automatically. Notably, recommender systems are a little different from general supervised learning tasks. In recommender systems, there are some factors (e.g., previous recommendation models or operation strategies of a online platform) that determine which items can be exposed to each individual user. Normally, the previous exposure results are not only relevant to the instances' features (i.e., user or item), but also affect their feedback ratings, thus leading to confounding bias in the recommendation models. To mitigate this bias, researchers have already provided a variety of strategies. However, there are still two issues that are underappreciated: 1) previous debiased RS approaches cannot effectively capture recommendation-specific, exposure-specific and their common knowledge simultaneously; 2) the true exposure results of the user-item pairs are partially inaccessible, so there would be some noises if we use their observability to approximate it as existing approaches. Motivated by this, we develop a novel debiasing recommendation approach. More specifically, we first propose a mutual information-based counterfactual learning framework based on the causal relationship among the instance features, exposure status, and ratings. This framework can 1) capture recommendation-specific, exposure-specific and their common knowledge by explicitly modeling the relationship among the causal factors, and 2) achieve robustness towards partially inaccessible exposure results by a pairwise learning strategy. Under such a framework, we implement an optimizable loss function with theoretical analysis. By minimizing this loss, we expect to obtain an unbiased recommendation model that reflects the users' real interests. Meanwhile, we also prove that our loss function has robustness towards the partial inaccessibility of the exposure status. Finally, extensive experiments on public datasets manifest the superiority of our proposed method in boosting the recommendation performance.

Robust Collaborative Filtering to Popularity Distribution Shift

In leading collaborative filtering (CF) models, representations of users and items are prone to learn popularity bias in the training data as shortcuts. The popularity shortcut tricks are good for in-distribution (ID) performance but poorly generalized to out-of-distribution (OOD) data, i.e., when popularity distribution of test data shifts w.r.t. the training one. To close the gap, debiasing strategies try to assess the shortcut degrees and mitigate them from the representations. However, there exist two deficiencies: (1) when measuring the shortcut degrees, most strategies only use statistical metrics on a single aspect (i.e., item frequency on item and user frequency on user aspect), failing to accommodate the compositional degree of a user–item pair; (2) when mitigating shortcuts, many strategies assume that the test distribution is known in advance. This results in low-quality debiased representations. Worse still, these strategies achieve OOD generalizability with a sacrifice on ID performance. In this work, we present a simple yet effective debiasing strategy, PopGo , which quantifies and reduces the interaction-wise popularity shortcut without any assumptions on the test data. It first learns a shortcut model, which yields a shortcut degree of a user–item pair based on their popularity representations. Then, it trains the CF model by adjusting the predictions with the interaction-wise shortcut degrees. By taking both causal- and information-theoretical looks at PopGo, we can justify why it encourages the CF model to capture the critical popularity-agnostic features while leaving the spurious popularity-relevant patterns out. We use PopGo to debias two high-performing CF models (matrix factorization [ 28 ] and LightGCN [ 19 ]) on four benchmark datasets. On both ID and OOD test sets, PopGo achieves significant gains over the state-of-the-art debiasing strategies (e.g., DICE [ 71 ] and MACR [ 58 ]). Codes and datasets are available at https://github.com/anzhang314/PopGo .

HN-GCCF: High-order neighbor-enhanced graph convolutional collaborative filtering

Graph convolutional collaborative filtering (CF) has recently made remarkable progress. Its great success is attributed to the embedding propagation mechanism of graph convolutional networks (GCNs), which can capture the collaborative signals hidden in the high-order neighbors by stacking multiple embedding propagation layers (i.e., graph convolution layers). However, we argue that existing GCN-based CF models are insufficient for exploiting collaborative filtering effects since their embedding propagation schemes and sampling strategies cannot make full use of the rich collaborative signals of the high-order neighbors. In this work, we propose two high-order neighbor-enhanced (HN) strategies from two different perspectives, i.e., embedding propagation (EP) and positive user–item pair (PP) construction. Specifically, the EP-based HN strategy designs a new embedding propagation scheme, i.e., a high-order graph convolution operation, to alleviate the excessive decay of high-order neighbor information. The PP-based HN strategy mines high-quality positive user–item pairs to enhance model training by utilizing users’ high-order neighbors as potentially positive samples. Besides, we present the high-order neighbor-enhanced graph convolutional collaborative filtering (HN-GCCF) model, equipped with the above two HN strategies. Extensive experiments demonstrate that HN-GCCF significantly outperforms state-of-the-art GCN-based CF models in terms of both effectiveness and efficiency.

Predicting Online Item-Choice Behavior: A Shape-Restricted Regression Approach

This paper examines the relationship between user pageview (PV) histories and their itemchoice behavior on an e-commerce website. We focus on PV sequences, which represent time series of the number of PVs for each user–item pair. We propose a shape-restricted optimization model that accurately estimates item-choice probabilities for all possible PV sequences. This model imposes monotonicity constraints on item-choice probabilities by exploiting partial orders for PV sequences, according to the recency and frequency of a user’s previous PVs. To improve the computational efficiency of our optimization model, we devise efficient algorithms for eliminating all redundant constraints according to the transitivity of the partial orders. Experimental results using real-world clickstream data demonstrate that our method achieves higher prediction performance than that of a state-of-the-art optimization model and common machine learning methods.

Personalized recommendation via inductive spatiotemporal graph neural network

Graph neural network-based collaborative filtering methods have achieved excellent performance in recommender systems. However, previous works have primarily focused on representation using the entire graph or a single sampled subgraph, which fails to address the issue of noise introduced by prolonged ineffective interactions. Moreover, these approaches lack the capacity to model temporal information in an inductive manner, thereby limiting their ability to fully capture the evolving dynamic interests of users over time in a real world scenario. To address the aforementioned challenges, we propose a novel personalized inductive spatiotemporal graph neural network-based framework PistGNN. PistGNN extracts multiple temporal-aware subgraphs from user–item pairs to avoid long-time gap meaningless interactions, which are then fed into a spatiotemporal graph neural network module. It is worth noting that PistGNN trains the model only by relying on subgraphs extracted from bipartite graphs, without depending on global information, and is therefore capable of predicting for new users or items. Finally, we propose an attention-based meta-learning method to personalize the aggregation of subgraphs’ embeddings. Extensive experiments conducted on four real-world datasets demonstrate the superiority of PistGNN over both inductive and transductive baseline methods.

Evaluation of the Recommendation Effect of Individual Interventions in a Recommender System

Collaborative filtering is a recommendation model that evaluates and recommends items that match the preferences of each user and with high purchase probability. However, some items are regularly purchased and do not necessarily require recommendation. Thus, identifying items with a high recommendation effect is one of the challenges when using recommendation models. The recommendation effect can be regarded as an intervention effect in causal inference by considering the recommendation of an item as intervention. The intervention effect of individual users can be estimated using models such as counterfactual regression (CFR). Nonetheless, because this model can only manage one type of intervention, it cannot be easily and directly applied to recommender systems when many recommended items appear as intervention. In this study, we extend this model to estimate the effects of individual interventions in multiple interventions using a single CFR by combining user and item features to form covariates of user–item pairs, thereby allowing the recommendation effects to be estimated for all user–item pairs. We demonstrate the effectiveness of the proposed method through experiments using artificial data.

SelfCF: A Simple Framework for Self-supervised Collaborative Filtering

Collaborative filtering (CF) is widely used to learn informative latent representations of users and items from observed interactions. Existing CF-based methods commonly adopt negative sampling to discriminate different items. That is, observed user-item pairs are treated as positive instances; unobserved pairs are considered as negative instances and are sampled under a defined distribution for training. Training with negative sampling on large datasets is computationally expensive. Further, negative items should be carefully sampled under the defined distribution, in order to avoid selecting an observed positive item in the training dataset. Unavoidably, some negative items sampled from the training dataset could be positive in the test set. Recently, self-supervised learning (SSL) , has emerged as a powerful tool to learn a model without negative samples. In this paper, we propose a self-supervised collaborative filtering framework (SelfCF) , that is specially designed for recommender scenario with implicit feedback. The proposed SelfCF framework simplifies Siamese networks and can be easily applied to existing deep-learning based CF models, which we refer to as backbone networks. The main idea of SelfCF is to augment the latent embeddings generated by backbone networks instead of the raw input of user/item ids. We propose and study three embedding perturbation techniques that can be applied to different types of backbone networks including both traditional CF models and graph-based models. The framework enables learning informative representations of users and items without negative samples, and is agnostic to the encapsulated backbones. We conduct experimental comparisons on four datasets, one self-supervised framework, and eight baselines to show that our framework may achieve even better recommendation accuracy than the encapsulated supervised counterpart with a 2×–4× faster training speed. The results also demonstrate that SelfCF can boost up the accuracy of a self-supervised framework BUIR by 17.79% on average and shows competitive performance with baselines.

On the Relationship between Explanation and Recommendation: Learning to Rank Explanations for Improved Performance

Explaining to users why some items are recommended is critical, as it can help users to make better decisions, increase their satisfaction, and gain their trust in recommender systems (RS). However, existing explainable RS usually consider explanation as a side output of the recommendation model, which has two problems: (1) It is difficult to evaluate the produced explanations, because they are usually model-dependent, and (2) as a result, how the explanations impact the recommendation performance is less investigated. In this article, explaining recommendations is formulated as a ranking task and learned from data, similarly to item ranking for recommendation. This makes it possible for standard evaluation of explanations via ranking metrics (e.g., Normalized Discounted Cumulative Gain). Furthermore, this article extends traditional item ranking to an item–explanation joint-ranking formalization to study if purposely selecting explanations could reach certain learning goals, e.g., improving recommendation performance. A great challenge, however, is that the sparsity issue in the user-item-explanation data would be inevitably severer than that in traditional user–item interaction data, since not every user–item pair can be associated with all explanations. To mitigate this issue, this article proposes to perform two sets of matrix factorization by considering the ternary relationship as two groups of binary relationships. Experiments on three large datasets verify the solution’s effectiveness on both explanation ranking and item recommendation.

KR-GCN: Knowledge-Aware Reasoning with Graph Convolution Network for Explainable Recommendation

Incorporating knowledge graphs (KGs) into recommender systems to provide explainable recommendation has attracted much attention recently. The multi-hop paths in KGs can provide auxiliary facts for improving recommendation performance as well as explainability. However, existing studies may suffer from two major challenges: error propagation and weak explainability. Considering all paths between every user-item pair might involve irrelevant ones, which leads to error propagation of user preferences. Defining meta-paths might alleviate the error propagation, but the recommendation performance would heavily depend on the pre-defined meta-paths. Some recent methods based on graph convolution network (GCN) achieve better recommendation performance, but fail to provide explainability. To tackle the above problems, we propose a novel method named K nowledge-aware R easoning with G raph C onvolution N etwork (KR-GCN). Specifically, to alleviate the effect of error propagation, we design a transition-based method to determine the triple-level scores and utilize nucleus sampling to select triples within the paths between every user-item pair adaptively. To improve the recommendation performance and guarantee the diversity of explanations, user-item interactions and knowledge graphs are integrated into a heterogeneous graph, which is performed with the graph convolution network. A path-level self-attention mechanism is adopted to discriminate the contributions of different selected paths and predict the interaction probability, which improves the relevance of the final explanation. Extensive experiments conducted on three real-world datasets show that KR-GCN consistently outperforms several state-of-the-art baselines. And human evaluation proves the superiority of KR-GCN on explainability.

More Efficient Secure Matrix Multiplication for Unbalanced Recommender Systems

With recent advances in homomorphic encryption (HE), it becomes feasible to run non-interactive machine learning (ML) algorithms on encrypted data without decryption. In this work, we propose novel encoding methods to pack matrix in a compact way and more efficient methods to perform matrix multiplication on homomorphically encrypted data, leading to a speed boost of <inline-formula><tex-math notation="LaTeX">$1.5\times - 20\times$</tex-math></inline-formula> for slim rectangular matrix multiplication compared with state-of-the-art. Moreover, we integrate our optimized secure matrix arithmetic with the MPI distributed computing framework, achieving scalable parallel secure matrix computation. Equipped with the optimized matrix multiplication, we propose <i>uSCORE</i> , a privacy-preserving cross-domain recommendation system for the unbalanced scenario, where a big data owner provides recommendation as a service to a client who has less data and computation power. Our design delegates most of the computation to the service provider, and has a low communication cost, which previous works failed to achieve. For a client who has 16 million user-item pairs to update, it only needs about 3 minutes (in the LAN setting) to prepare the encrypted data. The server can finish the update process on the encrypted data in less than half an hour, effectively reducing the client's test error from 0.72 to 0.62.

LightAD: accelerating AutoDebias with adaptive sampling

In recommendation systems, bias is ubiquitous because the data are collected from user behaviors rather than from reasonable experiments. AutoDebias, which resorts to metalearning to find appropriate debiasing configurations, i.e., pseudolabels and confidence weights for all user-item pairs, has been demonstrated to be a generic and effective solution for tackling various biases. Nevertheless, setting pseudolabels and weights for every user-item pair can be a time-consuming process. Therefore, AutoDebias suffers from an enormous computational cost, making it less applicable to real cases. Although stochastic gradient descent with a uniform sampler can be applied to accelerate training, this approach significantly deteriorates model convergence and stability. To overcome this problem, we propose LightAutoDebias (short as LightAD), which equips AutoDebias with a specialized importance sampling strategy. The sampler can adaptively and dynamically draw informative training instances, which results in better convergence and stability than does the standard uniform sampler. Several experiments on three benchmark datasets validate that our LightAD accelerates AutoDebias by several magnitudes while maintaining almost equal accuracy.

Dual-track feedback aggregation recommendation model for programming training

Programming online judges (POJs) are widely used to train programming skills, and exercise recommendation algorithms in POJs have attracted wide attention. The current programming recommendation algorithms cannot make full use of the feedback of user–item pairs and cannot effectively express students’ mastery of exercises. Therefore, we propose a dual-track feedback aggregation recommendation model for programming training (DTFARec). In this model, multiple types of feedback fusion mechanism (MTFFM) and dual-track method (DTM) are proposed to solve this problem and can better express students’ mastery of exercises. The MTFFM uses an attention mechanism to learn different feedback information, and the DTM is able to fuse information from both feedback and interactive aspects. The experimental results on a real-world dataset show that the model has better recommendation performance than the best performing benchmark and that our method can effectively model students’ mastery of exercises.

Deep Rating and Review Neural Network for Item Recommendation.

To alleviate the sparsity issue, many recommender systems have been proposed to consider the review text as the auxiliary information to improve the recommendation quality. Despite success, they only use the ratings as the ground truth for error backpropagation. However, the rating information can only indicate the users' overall preference for the items, while the review text contains rich information about the users' preferences and the attributes of the items. In real life, reviews with the same rating may have completely opposite semantic information. If only the ratings are used for error backpropagation, the latent factors of these reviews will tend to be consistent, resulting in the loss of a large amount of review information. In this article, we propose a novel deep model termed deep rating and review neural network (DRRNN) for recommendation. Specifically, compared with the existing models that adopt the review text as the auxiliary information, DRRNN additionally considers both the target rating and target review of the given user-item pair as ground truth for error backpropagation in the training stage. Therefore, we can keep more semantic information of the reviews while making rating predictions. Extensive experiments on four publicly available datasets demonstrate the effectiveness of the proposed DRRNN model in terms of rating prediction.

Cross-Domain Explicit–Implicit-Mixed Collaborative Filtering Neural Network

Collaborative filtering (CF) is a classical model for recommender systems. Though neural network-based CF models have been shown to perform well in some cases, they still suffer from the sparsity and cold-start issues. To address these issues, we propose a novel neural network-based CF model, termed the cross-domain explicit–implicit-mixed CF neural network (CEICFNet). The proposed model utilizes deep neural networks to learn latent factors not only from the explicit ratings and the implicit interactions but also in a cross-domain manner. In particular, domain-shared multilayer perception (MLP) networks are designed to learn the user rating latent factors and the user interaction latent factors from the explicit ratings and the implicit interactions, respectively, which act as bridges for transferring knowledge across domains. On the other hand, domain-specific MLP networks are designed to learn the item rating latent factors and the item interaction latent factors from the explicit ratings and the implicit interactions, respectively. Then, in each domain, based on the user rating latent factors and the item rating latent factors, the rating predictive representation for each user–item pair can be learned by an MLP. Similarly, the interaction predictive representation for each user–item pair can be learned. For integrating the explicit ratings and the implicit interactions, in each domain, a fully connected layer is used to automatically assign different weights to the rating predictive representations and the interaction predictive representations, based on which the final interaction probability can be generated. Extensive experiments are conducted on five datasets and the results have confirmed the effectiveness of our model.

Deep Collaborative Recommendation Algorithm Based on Attention Mechanism

Representation learning-based collaborative filtering (CF) methods address the linear relationship of user-items with dot products and cannot study the latent nonlinear relationship applied to implicit feedback. Matching function learning-based CF methods directly learn the complicated mapping functions that map user-item pairs to matching scores, which has limitations in identifying low-rank relationships. To this end, we propose a deep collaborative recommendation algorithm based on attention mechanism (DACR). First, before the user-item representations are input into the DNNs, we utilize the attention mechanism to adaptively assign different weights to the user-item representations, which captures the hidden information in implicit feedback. After that, we input the user-item representations with corresponding weights into the representation learning and matching function learning modules. Finally, we concatenate the prediction vectors learned from different dimensions to predict the matching scores. The results show that we can improve the expression ability of the model while taking into account not only the nonlinear information hidden in implicit feedback, but also the low-rank relationships of user-item pairs to obtain more accurate predictions. Through detailed experiments on two datasets, we find that the ranking capability of the DACR model is enhanced compared with other baseline models, and the evaluation metrics HR and NDCG of DACR are increased by 0.88–1.19% and 0.65–1.15%, respectively.

Attention on Global–Local Representation Spaces in Recommender Systems

In this study, we present a novel clustering-based collaborative filtering (CF) method for recommender systems. Clustering-based CF methods can effectively deal with data sparsity and scalability problems. However, most of them are applied to a single representation space, which might not characterize complex user–item interactions well. We argue that the user–item interactions should be observed from multiple views and characterized in an adaptive way. To address this issue, we leveraged the global and local properties to construct multiple representation spaces by learning various training datasets and loss functions. An attention network was built to generate a blended representation according to the relative importance of the representation spaces for each user–item pair, providing a flexible way to characterize diverse user–item interactions. Substantial experiments were evaluated on four popular benchmark datasets. The results show that the proposed method is superior to several CF methods where only one representation space is considered.

Siamese Graph-Based Dynamic Matching for Collaborative Filtering

Behaviorally similar neighbors in the interaction graph have been actively explored to facilitate the collaboration between users and items and address the interaction sparsity issue. We investigate homogenous neighbors between users or items to mine collaborative signals for embedding learning. In the case of multiple and complex composition of user interests and item attributes, traditional uniform embedding is insufficient to depict matching between a specific user-item pair. Therefore, we propose a siamese graph-based dynamic matching (SGDM) model for collaborative filtering. A target-aware dual attention module is introduced to update neighbors with their varying contributions to unveil users’ interests, which explicitly encodes the clue of the candidate matching target. For a target user-item pair, homogeneous neighbors participate in the interest propagation through graph convolution, which learns user/item embeddings respectively on the siamese homogeneous graphs. The dual dynamic aggregation in graph convolution endows a specific user-item pair with dynamic matching, which is expected to meet the needs of fine-grained filtering and promote the performance of collaborative filtering. Extensive experimental results confirm the collaboration between siamese homogeneous graphs of users and items. It further illustrates the effectiveness of the proposed SGDM in mining homogeneous collaborative signals for embedding learning and collaborative filtering.

LightFIG: simplifying and powering feature interactions via graph for recommendation.

The attributes of users and items contain key information for recommendation. The latest advances demonstrate that better representations can be learned by performing graph convolutions on attribute graph of the user-item pair. Recently proposed models construct graphs that not only connect edges between user attributes and item attributes, but also within user (item) attributes. However, to determine whether a user is interested in an item, the relationships between user attributes and item attributes are what we really need to mine. In many cases, due to the low correlation between relationships within user (item) attributes and preference of the user, the artificially connected edges within user (item) attributes contribute little to the recommendation. Even worse, including them will not only drastically increase the training time, but may also introduce noise and lead to degraded performance. In addition, the use of the optimizer is also relatively simple. One single optimizer is the default configuration for recommendation models. This may not be the best way to exploit it in many cases however. To solve these problems, we propose an enhanced model named LightFIG in this work. The key idea of LightFIG is twofold: First, we simplify the construction of attribute graph which focuses on mining relationships cross user attributes and item attributes, not between user (item) attributes. Second, we propose the idea of relay optimization, which employs two different optimizers to continuously optimize model parameters. Comprehensive experiments on three public datasets demonstrate the effectiveness of our proposed model.