User Preference Model Research Articles

Multimodal Recommender Systems: A Survey

The recommender system (RS) has been an integral toolkit of online services. They are equipped with various deep learning techniques to model user preference based on identifier and attribute information. With the emergence of multimedia services, such as short videos, news and etc. , understanding these contents while recommending becomes critical. Besides, multimodal features are also helpful in alleviating the problem of data sparsity in RS. Thus, M ultimodal R ecommender S ystem (MRS) has attracted much attention from both academia and industry recently. In this paper, we will give a comprehensive survey of the MRS models, mainly from technical views. First, we conclude the general procedures and major challenges for MRS. Then, we introduce the existing MRS models according to four categories, i.e., Modality Encoder , Feature Interaction , Feature Enhancement and Model Optimization . Besides, to make it convenient for those who want to research this field, we also summarize the dataset and code resources. Finally, we discuss some promising future directions of MRS and conclude this paper. To access more details of the surveyed papers, such as implementation code, we open source a repository.

Efficient and adaptive secure cross-domain recommendations

In recommendation systems, cross-domain recommendation (CDR) has emerged as an effective method for solving the problem of data sparsity and cold start. However, challenges such as data heterogeneity, data security, and recommendation accuracy are faced by existing CDR methods. An efficient and adaptive secure cross-domain recommendation (EAAS-CDR) model is proposed. First, the Automatic Encoder–Decoder (AutoED) model is proposed to address variations in user–item interaction quantity and item characteristics across various domains. The AutoED model enables adaptive and secure cross-domain recommendations by modeling user preferences across various domains, unifying vector dimensions, and avoiding the release of the rating matrix. An efficient cross-domain privacy protection method is proposed to resist data leakage. To ensure feature invariance for knowledge transfer, the inherent low perturbation characteristics of the Johnson–Lindenstrauss Transform mechanism are leveraged and combined with the output of our encoder. Finally, domain relevance measurement metrics and a domain fusion algorithm are proposed to use the source domain knowledge in the target domain and improve the recommendation accuracy of the model. Experimental results demonstrate that our approach is not only more efficient but also more secure than existing CDR approaches.

Deconfounding User Preference in Recommendation Systems through Implicit and Explicit Feedback

Recommender systems are influenced by many confounding factors (i.e., confounders) which result in various biases (e.g., popularity biases) and inaccurate user preference. Existing approaches try to eliminate these biases by inference with causal graphs. However, they assume all confounding factors can be observed and no hidden confounders exist. We argue that many confounding factors (e.g., season) may not be observable from user–item interaction data, resulting inaccurate user preference. In this article, we propose a deconfounded recommender considering unobservable confounders. Specifically, we propose a new causal graph with explicit and implicit feedback, which can better model user preference. Then, we realize a deconfounded estimator by the front-door adjustment, which is able to eliminate the effect of unobserved confounders. Finally, we conduct a series of experiments on two real-world datasets, and the results show that our approach performs better than other counterparts in terms of recommendation accuracy.

Personalized Recommendation Systems Powered By Large Language Models: Integrating Semantic Understanding and User Preferences

This study proposes a novel personalized recommendation system leveraging Large Language Models (LLMs) to integrate semantic understanding with user preferences [1]. The system addresses critical challenges in traditional recommendation approaches by harnessing LLMs' advanced natural language processing capabilities. We introduce a framework combining a fine-tuned Roberta semantic analysis model with a multi-modal user preference extraction mechanism.The LLM component undergoes domain adaptation using Masked Language Modeling on a corpus of 112,000 user reviews from the MyAnimeList dataset, followed by task-specific fine-tuning using contrastive learning. User preferences are modeled through a weighted combination of explicit ratings, review sentiment, and implicit feedback, incorporating temporal dynamics through a time-decay function. Experimental results demonstrate significant improvements over state-of-the-art baselines, including Matrix Factorization, Neural Collaborative Filtering, BERT4Rec, and LightGCN. Our LLM-powered system achieves an 8.6%increase in NDCG@10 and a 10.5% improvement in Mean Reciprocal Rank compared to the best-performing baseline. Ablation studies reveal the synergistic effect of integrating LLM-based semantic understanding with user preference modeling. Case studies highlight the system's ability to recommend long-tail items and provide cross-genre suggestions, showcasing its capacity for nuanced content understanding. Scalability analysis indicates that while the LLM-based approach has higher initial computational costs, its performance scales comparably to other deep learning models for larger datasets. This research contributes to the field by demonstrating the effectiveness of LLMs in enhancing recommendation accuracy and diversity. Future work will explore advanced LLM architectures, multi-modal data integration, and techniques to improve computational efficiency and interpretability of recommendations.

Knowledge Graph Enhanced Contextualized Attention-Based Network for Responsible User-Specific Recommendation

With ever-increasing dataset size and data storage capacity, there is a strong need to build systems that can effectively utilize these vast datasets to extract valuable information. Large datasets often exhibit sparsity and pose cold start problems, necessitating the development of responsible recommender systems. Knowledge graphs have utility in responsibly representing information related to recommendation scenarios. However, many studies overlook explicitly encoding contextual information, which is crucial for reducing the bias of multi-layer propagation. Additionally, existing methods stack multiple layers to encode high-order neighbor information while disregarding the relational information between items and entities. This oversight hampers their ability to capture the collaborative signal latent in user-item interactions. This is particularly important in health informatics, where knowledge graphs consist of various entities connected to items through different relations. Ignoring the relational information renders them insufficient for modeling user preferences. This work presents an end-to-end recommendation framework named KGCAN (Knowledge Graph Enhanced Contextualized Attention-Based Network), which explicitly encodes both relational and contextual information of entities to preserve the original entity information. Furthermore, a user-specific attention mechanism is employed to capture personalized recommendations. The proposed model is validated on three benchmark datasets through extensive experiments. The experimental results demonstrate that KGCAN outperforms existing knowledge graph based recommendation models. Additionally, a case study from the healthcare domain is discussed, highlighting the importance of attention mechanisms and high-order connectivity in the responsible recommendation system for health informatics.

Modeling multi-factor user preferences based on Transformer for next point of interest recommendation

With the rapid development of the mobile network and the gradual popularization of mobile devices, more and more users try to find attractive places to visit through WeChat, Twitter applications. In this trend, personalized next point of interest(POI) recommendation in the Location-based Social Network (LBSN) has become the focus of research and practice. Most existing studies capture user interest changes between different days (i.e. weekend and weekday), however, they ignore seasonal factors in time transition and category factors and thus fail to capture seasonal-level and category-level movement patterns in users’ mobile trajectories. Besides, they neglect the relevance between POIs from all users’ trajectory and fail to generate expressive POI embedding representation without constructing trajectory graph, which will reduce the accuracy of the next POI recommendation. To address the issues above, a next POI recommendation method for modeling Multi-factor User Preferences based on Transformer (MUPT) is developed, which consists of a global POI relationship modeling, a local multi-factor user preference modeling and a prediction module. It first learns the collaborative information of users with similar behavior to generate expressive POI embedding representation. Then it captures the personalized movement patterns of users at the POI, category and time levels based on Transformer mechanism in the local module. Especially the seasonal and other fine-grained information on the time series are learned in the time preference modeling part. The prediction module designed tracks the relationship between multi-level motion pattern representation of user check-in behavior and the next POI accessed by the user, and it finally obtains user’s preference probability for next POIs. An extensive experiment has been conducted on four datasets, and the experimental results analysis demonstrates that our proposed MUPT method is superior to other methods in terms of accuracy(ACC), mean reciprocal rank(MRR) and normalized discounted cumulative gain(NDCG).

Explicitly Exploiting Implicit User and Item Relations in Graph Convolutional Network (GCN) for Recommendation

Most existing collaborative filtering-based recommender systems rely solely on available user–item interactions for user and item representation learning. Their performance often suffers significantly when interactions are sparse, as limited user and item interactions are insufficient for learning robust representations. To address this issue, recent research has explored additional information between users and items by leveraging the user–item bipartite graph. However, these methods have not fully exploited high-order neighborhood information, primarily using sampled interactions to enrich training data rather than integrating this information directly into representation learning. In this paper, we propose a novel model, EIR-GCN (Embedding Integration with Relational Graph Convolutional Network), which directly incorporates various types of collaborative relations, such as user–user and item–item interactions, into the embedding function for user preference modeling. Specifically, our model employs advanced graph convolutional network (GCN) techniques to integrate user–item, user–user, and item–item relations for comprehensive representation learning. EIR-GCN initially selects the most influential second-order neighbors from the user–item bipartite graph to form user–user and item–item connections. With these enriched connections, a message-passing method is adopted to learn node representations by aggregating messages from directly linked nodes, including first-order item neighbors and selected second-order user neighbors. Extensive experiments on several public datasets demonstrate that EIR-GCN outperforms strong baselines, including recent GCN-based models and those exploiting high-order information. Our results show that EIR-GCN achieves state-of-the-art performance and effectively addresses the sparsity issue, highlighting its robustness and efficacy in recommendation tasks.

KGR: A Kernel-Mapping Based Group Recommender System Using Trust Relations

A massive amount of information explosion over the internet has caused a possible difficulty of information overload. To overcome this, Recommender systems are systematic tools that are rapidly being employed in several domains such as movies, travel, E-commerce, and music. In the existing research, several methods have been proposed for single-user modeling, however, the massive rise of social connections potentially increases the significance of group recommender systems (GRS). A GRS is one that jointly recommends a list of items to a collection of individuals based on their interests. Moreover, the single-user model poses several challenges to recommender systems such as data sparsity, cold start, and long tail problems. On the contrary hand, another hotspot for group-based recommendation is the modeling of user preferences and interests based on the groups to which they belong using effective aggregation strategies. To address such issues, a novel “KGR” group recommender system based on user-trust relations is proposed in this study using kernel mapping techniques. In the proposed model, user-trust networks or relations are exploited to generate trust-based groups of users which is one of the important behavioral and social aspects. More precisely, in KGR the group kernels and group residual matrices are exploited as well as seeking a multi-linear mapping between encoded vectors of group-item interactions and probability density function indicating how groups will rate the items. Moreover, to emphasize the relevance of individual preferences of users in a group to which they belong, a hybrid approach is also suggested in which group kernels and individual user kernels are merged as additive and multiplicative models. Furthermore, the proposed KGR is validated on two different trust-based datasets including Film Trust and CiaoDVD. In addition, KGR outperforms with an RMSE value of 0.3306 and 0.3013 on FilmTrust and CiaoDVD datasets which are lower than the 1.8176 and 1.1092 observed with the original KMR.

Aspect-enhanced Explainable Recommendation with Multi-modal Contrastive Learning

Explainable recommender systems ( ERS ) aim to enhance users’ trust in the systems by offering personalized recommendations with transparent explanations. This transparency provides users with a clear understanding of the rationale behind the recommendations, fostering a sense of confidence and reliability in the system’s outputs. Generally, the explanations are presented in a familiar and intuitive way, which is in the form of natural language, thus enhancing their accessibility to users. Recently, there has been an increasing focus on leveraging reviews as a valuable source of rich information in both modeling user-item preferences and generating textual interpretations, which can be performed simultaneously in a multi-task framework. Despite the progress made in these review-based recommendation systems, the integration of implicit feedback derived from user-item interactions and user-written text reviews has yet to be fully explored. To fill this gap, we propose a model named SERMON (A s pect-enhanced E xplainable R ecommendation with M ulti-modal C o ntrast Lear n ing). Our model explores the application of multimodal contrastive learning to facilitate reciprocal learning across two modalities, thereby enhancing the modeling of user preferences. Moreover, our model incorporates the aspect information extracted from the review, which provides two significant enhancements to our tasks. Firstly, the quality of the generated explanations is improved by incorporating the aspect characteristics into the explanations generated by a pre-trained model with controlled textual generation ability. Secondly, the commonly used user-item interactions are transformed into user-item-aspect interactions, which we refer to as interaction triple, resulting in a more nuanced representation of user preference. To validate the effectiveness of our model, we conduct extensive experiments on three real-world datasets. The experimental results show that our model outperforms state-of-the-art baselines, with a 2.0% improvement in prediction accuracy and a substantial 24.5% enhancement in explanation quality for the TripAdvisor dataset.

Enhancing conversational recommendation systems through the integration of KNN with ConLinUCB contextual bandits

In recommender system research, contextual multi-armed bandits have shown promise in delivering tailored recommendations by utilizing contextual data. However, their effectiveness is often curtailed by the cold start problem, arising from the lack of initial user data. This necessitates extensive exploration to ascertain user preferences, consequently impeding the speed of learning. The advent of conversational recommendation systems offers a solution. Through these systems, the conversational contextual bandit algorithm swiftly learns user preferences for specific key-terms via interactive dialogues, thereby enhancing the learning pace. Despite these advancements, there are limitations in current methodologies. A primary issue is the suboptimal integration of data from key-term-centric dialogues and arm-level recommendations, which could otherwise expedite the learning process. Another crucial aspect is the strategic suggestion of exploratory key phrases. These phrases are essential in quickly uncovering users potential interests in various domains, thus accelerating the convergence of accurate user preference models. Addressing these challenges, the ConLinUCB framework emerges as a groundbreaking solution. It ingeniously combines feedback from both arm-level and key-term-level interactions, significantly optimizing the learning trajectory. Building upon this, the framework integrates a K-nearest neighbour (KNN) approach to refine key-term selection and arm recommendations. This integration hinges on the similarity of user preferences, further hastening the convergence of the parameter vectors.

Color Theme Evaluation through User Preference Modeling

Color composition (or color theme) is a key factor to determine how well a piece of art work or graphical design is perceived by humans. Despite a few color harmony models have been proposed, their results are often less satisfactory since they mostly neglect the variations of aesthetic cognition among individuals and treat the influence of all ratings equally as if they were all rated by the same anonymous user. To overcome this issue, in this paper we propose a new color theme evaluation model by combining a back propagation neural network and a kernel probabilistic model to infer both the color theme rating and the user aesthetic preference. Our experiment results show that our model can predict more accurate and personalized color theme ratings than state of the art methods. Our work is also the first-of-its-kind effort to quantitatively evaluate the correlation between user aesthetic preferences and color harmonies of five-color themes, and study such a relation for users with different aesthetic cognition.

Group-aware graph neural networks for sequential recommendation

Sequential recommendations play a crucial role in recommender systems. Existing methods commonly focus on extracting sequential patterns within individual item sequences to make personalized recommendations. However, this paper argues that such a closure strategy has limitations. Firstly, the inherent uncertainty and sparsity within these sequences pose considerable challenges in accurately capturing sequential patterns. Secondly, ignoring group behaviors beyond the sequence hinders the modeling of user similarities, which are essential for recommendations. To address these limitations, this paper proposes a novel collaborative sequential recommendation approach, called Group-aware Graph Neural Networks (GaGNN). GaGNN seeks to incorporate group behaviors within sequential recommendations, thereby enhancing the modeling of user preferences through the assimilation of collective wisdom. GaGNN constructs two transition graphs, each offering a distinct behavioral perspective. The group transition graph is designed to capture the collaborative effects and learn collaborative information, whereas the individual transition graph aims to articulate individual behaviors, thereby extracting sequential information. Additionally, a multi-source information fusion module is designed to bridge semantic gaps between the two types of information and yield a holistic understanding of user preferences. Through rigorous experimental comparisons and ablation studies, the superiority of GaGNN and the beneficial integration of collaborative information are emphatically validated.

Multi-view contrastive learning with virtual social group influence for social recommendation

Social recommendation systems leverage user–item interaction and user–user social network data to model user preferences and provide recommendations. Previous research has shown that capturing the influence of high-order neighbors and social groups in social networks can help model user preferences. However, challenges still exist in identifying important high-order neighbors from many candidates and modeling social group influence without direct information on social groups. This paper proposes a Multi-view Contrastive Learning with Social Group Influence (MCLSGI) method to address these challenges. Our approach uses a graph walk method to identify users’ virtual social groups and a GNN-based framework to model user preferences in different views, capturing the influence of direct and high-order neighbors as well as social groups. We also adopt multi-view contrastive learning to fuse users’ preferences in different views. We conducted experiments on two real-world datasets (Ciao and Epinions) to validate our method’s effectiveness. Compared to the best baseline, we improved by 1.39% and 1.21% with MAE, and 1.79% and 0.96% with RMSE.

Continuous-Time User Preference Modelling for Temporal Sets Prediction

Continuous-Time User Preference Modelling for Temporal Sets Prediction

FacetCRS: Multi-Faceted Preference Learning for Pricking Filter Bubbles in Conversational Recommender System

The filter bubble is a notorious issue in Recommender Systems (RSs), which describes the phenomenon whereby users are exposed to a limited and narrow range of information or content that reinforces their existing dominant preferences and beliefs. This results in a lack of exposure to diverse and varied content. Many existing works have predominantly examined filter bubbles in static or relatively-static recommendation settings. However, filter bubbles will be continuously intensified over time due to the feedback loop between the user and the system in the real-world online recommendation. To address these issues, we propose a novel paradigm, Multi-Facet Preference Learning for Pricking Filter Bubbles in Conversational Recommender System (FacetCRS), which aims to burst filter bubbles in the conversational recommender system (CRS) through timely user-item interactions via natural language conversations. By considering diverse user preferences and intentions, FacetCRS automatically model user preference into multi-facets, including entity-, word-, context-, and review-facet, to capture diverse and dynamic user preferences to prick filter bubbles in the CRS. It is an end-to-end CRS framework to adaptively learn representations of various levels of preference facet and diverse types of external knowledge. Extensive experiments on two publicly available benchmark datasets demonstrate that our proposed method achieves state-of-the-art performance in mitigating filter bubbles and enhancing recommendation quality in CRS.

Learning Accurate and Bidirectional Transformation via Dynamic Embedding Transportation for Cross-Domain Recommendation

With the rapid development of Internet and Web techniques, Cross-Domain Recommendation (CDR) models have been widely explored for resolving the data-sparsity and cold-start problem. Meanwhile, most CDR models should utilize explicit domain-shareable information (e.g., overlapped users or items) for knowledge transfer across domains. However, this assumption may not be always satisfied since users and items are always non-overlapped in real practice. The performance of many previous works will be severely impaired when these domain-shareable information are not available. To address the aforementioned issues, we propose the Joint Preference Exploration and Dynamic Embedding Transportation model (JPEDET) in this paper which is a novel framework for solving the CDR problem when users and items are non-overlapped. JPEDET includes two main modules, i.e., joint preference exploration module and dynamic embedding transportation module. The joint preference exploration module aims to fuse rating and review information for modelling user preferences. The dynamic embedding transportation module is set to share knowledge via neural ordinary equations for dual transformation across domains. Moreover, we innovatively propose the dynamic transport flow equipped with linear interpolation guidance on barycentric Wasserstein path for achieving accurate and bidirectional transformation. Our empirical study on Amazon datasets demonstrates that JPEDET significantly outperforms the state-of-the-art models under the CDR setting.

MultiCBR: Multi-view Contrastive Learning for Bundle Recommendation

Bundle recommendation seeks to recommend a bundle of related items to users to improve both user experience and the profits of platform. Existing bundle recommendation models have progressed from capturing only user-bundle interactions to the modeling of multiple relations among users, bundles, and items. CrossCBR, in particular, incorporates cross-view contrastive learning into a two-view preference learning framework, significantly improving SOTA performance. It does, however, have two limitations: (1) the two-view formulation does not fully exploit all the heterogeneous relations among users, bundles, and items; and (2) the “early contrast and late fusion” framework is less effective in capturing user preference and difficult to generalize to multiple views. In this article, we present MultiCBR, a novel Multi -view C ontrastive learning framework for B undle R ecommendation. First, we devise a multi-view representation learning framework capable of capturing all the user-bundle, user-item, and bundle-item relations, especially better utilizing the bundle-item affiliations to enhance sparse bundles’ representations. Second, we innovatively adopt an “early fusion and late contrast” design that first fuses the multi-view representations before performing self-supervised contrastive learning. In comparison to existing approaches, our framework reverses the order of fusion and contrast, introducing the following advantages: (1) Our framework is capable of modeling both cross-view and ego-view preferences, allowing us to achieve enhanced user preference modeling; and (2) instead of requiring quadratic number of cross-view contrastive losses, we only require two self-supervised contrastive losses, resulting in minimal extra costs. Experimental results on three public datasets indicate that our method outperforms SOTA methods. The code and dataset can be found in the github repo https://github.com/HappyPointer/MultiCBR .

Multipath-guided heterogeneous graph neural networks for sequential recommendation

With the explosion of information and users’ changing interest, program sequential recommendation becomes increasingly important for TV program platforms to help their users find interesting programs. Existing sequential recommendation methods mainly focus on modeling user preferences from users’ historical interaction behaviors directly, with insufficient learning about the dynamics of programs and users, while ignoring the rich semantic information from the heterogeneous graph. To address these issues, we propose the multipath-guided heterogeneous graph neural networks for TV program sequential recommendation (MHG-PSR), which can enhance the representations of programs and users through multiple paths in heterogeneous graphs. In our method, the auxiliary information is fused to supplement the semantics of program and user to obtain initial representations. Then, we explore the interactive behaviors of programs and users with temporal and auxiliary information to model the collaborative signals in the heterogeneous graph and extract the users’ dynamic preferences of programs. Extensive experiments on real-world datasets verify the proposed method can effectively improve the performance of TV program sequential recommendation.

Meta-path aware dynamic graph learning for friend recommendation with user mobility

Recently, friend recommendation has gained widespread popularity in location-based social networks (LBSNs), which provides more opportunities for users to forge new friendships. Most existing studies exploit user trajectories or check-ins of Point-Of-Interests (POIs) to predict friendships based on geographic homophily. However, the dynamics of social relationships are left insufficiently considered in modeling user preferences. In this paper, we explore how geographical and social preferences influence each other in a dynamic manner. Specifically, we propose a Meta-path aware Dynamic Graph with Subgraph Inference, named MDyGSI, which models the evolution of user preferences with time-phased sequences of POIs and social relationships for friend recommendation in LBSNs. In each time step of the evolution, geographical and social preferences are modeled through behavior-specific meta-paths in a dynamic heterogeneous graph. The formations of different meta-paths are facilitate by each other to explore mutual influences of dual preferences. To keep the dynamics of social relationships aligned with check-in history, reliable User-User connections are sampled from social graphs based on geographical collaborative filtering in each step, which also avoids noisy social interactions. Furthermore, the dual preferences are concatenated with evolutionary weights measured by a Gated Recurrent Unit for final recommendation. Experimental results on two real-world datasets show significant improvements in MDyGSI over the state-of-the-art methods.

Real-time Multi-CNN-based Emotion Recognition System for Evaluating Museum Visitors’ Satisfaction

Conventional studies on the satisfaction of museum visitors focus on collecting information through surveys to provide a one-way service to visitors, and thus it is impossible to obtain feedback on the real-time satisfaction of visitors who are experiencing the museum exhibition program. In addition, museum practitioners lack research on automated ways to evaluate a produced content program's lifecycle and its appropriateness. To overcome these problems, we propose a novel multi-convolutional neural network, called VimoNet, which is able to recognize visitors emotions automatically in real-time based on their facial expressions and body gestures. Furthermore, we design a user preference model of content and a framework to obtain feedback on content improvement for providing personalized digital cultural heritage content to visitors. Specifically, we define seven emotions of visitors and build a dataset of visitor facial expressions and gestures with respect to the emotions. Using the dataset, we proceed with feature fusion of face and gesture images trained on the DenseNet-201 and VGG-16 models for generating a combined emotion recognition model. From the results of the experiment, VimoNet achieved a classification accuracy of 84.10%, providing 7.60% and 14.31% improvement, respectively, over a single face and body gesture-based method of emotion classification performance. It is thus possible to automatically capture the emotions of museum visitors via VimoNet, and we confirm its feasibility through a case study with respect to digital content of cultural heritage.