Interactive Recommender System Research Articles

Discovering Personalized Semantics for Soft Attributes in Recommender Systems Using Concept Activation Vectors

Interactive recommender systems have emerged as a promising paradigm to overcome the limitations of the primitive user feedback used by traditional recommender systems (e.g., clicks, item consumption, ratings). They allow users to express intent, preferences, constraints, and contexts in a richer fashion, often using natural language (including faceted search and dialogue). Yet more research is needed to find the most effective ways to use this feedback. One challenge is inferring a user’s semantic intent from the open-ended terms or attributes often used to describe a desired item. This is critical for recommender systems that wish to support users in their everyday, intuitive use of natural language to refine recommendation results. Leveraging concept activation vectors (CAVs) [ 26 ], a recently developed approach for model interpretability in machine learning, we develop a framework to learn a representation that captures the semantics of such attributes and connects them to user preferences and behaviors in recommender systems. One novel feature of our approach is its ability to distinguish objective and subjective attributes (both subjectivity of degree and of sense ) and associate different senses of subjective attributes with different users. We demonstrate on both synthetic and real-world datasets that our CAV representation not only accurately interprets users’ subjective semantics but also can be used to improve recommendations through interactive item critiquing .

Personalised Multi-modal Interactive Recommendation with Hierarchical State Representations

Multi-modal interactive recommender systems (MMIRS) can effectively guide users towards their desired items through multi-turn interactions by leveraging the users’ real-time feedback (in the form of natural-language critiques) on previously recommended items (such as images of fashion products). In this scenario, the users’ preferences can be expressed by both the users’ past interests from their historical interactions and their current needs from the real-time interactions. However, it is typically challenging to make satisfactory personalised recommendations across multi-turn interactions due to the difficulty in balancing the users’ past interests and the current needs for generating the users’ state (i.e., current preferences) representations over time. However, hierarchical reinforcement learning has been successfully applied in various fields by decomposing a complex task into a hierarchy of more easily addressed subtasks. In this journal article, we propose a novel personalised multi-modal interactive recommendation model (PMMIR) using hierarchical reinforcement learning to more effectively incorporate the users’ preferences from both their past and real-time interactions. In particular, PMMIR decomposes the personalised interactive recommendation process into a sequence of two subtasks with hierarchical state representations: a first subtask where a history encoder learns the users’ past interests with the hidden states of history for providing personalised initial recommendations and a second subtask where a state tracker estimates the current needs with the real-time estimated states for updating the subsequent recommendations. The history encoder and the state tracker are jointly optimised with a single objective by maximising the users’ future satisfaction with the recommendations. Following previous work, we train and evaluate our PMMIR model using a user simulator that can generate natural-language critiques about the recommendations as a surrogate for real human users. Experiments conducted on two derived fashion datasets from two well-known public datasets demonstrate that our proposed PMMIR model yields significant improvements in comparison to the existing state-of-the-art baseline models. The datasets and code are publicly available at: https://github.com/yashonwu/pmmir

Maximum Entropy Policy for Long-Term Fairness in Interactive Recommender Systems

Maximum Entropy Policy for Long-Term Fairness in Interactive Recommender Systems

Towards Simulation-Based Evaluation of Recommender Systems with Carousel Interfaces

Offline data-driven evaluation is considered a low-cost and more accessible alternative to the online empirical method of assessing the quality of recommender systems. Despite their popularity and effectiveness, most data-driven approaches are unsuitable for evaluating interactive recommender systems. In this article, we attempt to address this issue by simulating the user interactions with the system as a part of the evaluation process. Particularly, we demonstrate that simulated users find their desired item more efficiently when recommendations are presented as a list of carousels compared to a simple ranked list.

Towards interactive explanation-based nutrition virtual coaching systems

The awareness about healthy lifestyles is increasing, opening to personalized intelligent health coaching applications. A demand for more than mere suggestions and mechanistic interactions has driven attention to nutrition virtual coaching systems (NVC) as a bridge between human–machine interaction and recommender, informative, persuasive, and argumentation systems. NVC can rely on data-driven opaque mechanisms. Therefore, it is crucial to enable NVC to explain their doing (i.e., engaging the user in discussions (via arguments) about dietary solutions/alternatives). By doing so, transparency, user acceptance, and engagement are expected to be boosted. This study focuses on NVC agents generating personalized food recommendations based on user-specific factors such as allergies, eating habits, lifestyles, and ingredient preferences. In particular, we propose a user-agent negotiation process entailing run-time feedback mechanisms to react to both recommendations and related explanations. Lastly, the study presents the findings obtained by the experiments conducted with multi-background participants to evaluate the acceptability and effectiveness of the proposed system. The results indicate that most participants value the opportunity to provide feedback and receive explanations for recommendations. Additionally, the users are fond of receiving information tailored to their needs. Furthermore, our interactive recommendation system performed better than the corresponding traditional recommendation system in terms of effectiveness regarding the number of agreements and rounds.

CIPL: Counterfactual Interactive Policy Learning to Eliminate Popularity Bias for Online Recommendation.

Popularity bias, as a long-standing problem in recommender systems (RSs), has been fully considered and explored for offline recommendation systems in most existing relevant researches, but very few studies have paid attention to eliminate such bias in online interactive recommendation scenarios. Bias amplification will become increasingly serious over time due to the existence of feedback loop between the user and the interactive system. However, existing methods have only investigated the causal relations among different factors statically without considering temporal dependencies inherent in the online interactive recommendation system, making them difficult to be adapted to online settings. To address these problems, we propose a novel counterfactual interactive policy learning (CIPL) method to eliminate popularity bias for online recommendation. It first scrutinizes the causal relations in the interactive recommender models and formulates a novel temporal causal graph (TCG) to guide the training and counterfactual inference of the causal interactive recommendation system. Concretely, TCG is used to estimate the causal relations of item popularity on prediction score when the user interacts with the system at each time during model training. Besides, it is also used to remove the negative effect of popularity bias in the test stage. To train the causal interactive recommendation system, we formulated our CIPL by the actor-critic framework with an online interactive environment simulator. We conduct extensive experiments on three public benchmarks and the experimental results demonstrate that our proposed method can achieve the new state-of-the-art performance.

Knowledge-Enhanced Causal Reinforcement Learning Model for Interactive Recommendation

Owing to its inherently dynamic nature and economical training cost, offline reinforcement learning (RL) is typically employed to implement an interactive recommender system (IRS). A crucial challenge in offline RL-based IRSs is the data sparsity issue, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , it is hard to mine user preferences well from the limited number of user-item interactions. In this paper, we propose a knowledge-enhanced causal reinforcement learning model (KCRL) to mitigate data sparsity in IRSs. We make technical extensions to the offline RL framework in terms of the reward function and state representation. Specifically, we first propose a group preference-injected causal user model (GCUM) to learn user satisfaction ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , reward) estimation. We introduce beneficial group preference information, namely, the group effect, via causal inference to compensate for incomplete user interests extracted from sparse data. Then, we learn the RL recommendation policy with the reward given by the GCUM. We propose a knowledge-enhanced state encoder (KSE) to generate knowledge-enriched user state representations at each time step, which is assisted by a self-constructed user-item knowledge graph. Extensive experimental results on real-world datasets demonstrate that our model significantly outperforms the baselines.

RecHap: An Interactive Recommender System for Navigating a Large Number of Mid-Air Haptic Designs.

Designing haptics is a difficult task especially when the user attempts to design a sensation from scratch. In the fields of visual and audio design, designers often use a large library of examples for inspiration, supported by intelligent systems like recommender systems. In this work, we contribute a corpus of 10 000 mid-air haptic designs (500 hand-designed sensations augmented 20x to create 10 000), and we use it to investigate a novel method for both novice and experienced hapticians to use these examples in mid-air haptic design. The RecHap design tool uses a neural-network based recommendation system that suggests pre-existing examples by sampling various regions of an encoded latent space. The tool also provides a graphical user interface for designers to visualize the sensation in 3D view, select previous designs, and bookmark favourites, all while feeling designs in real-time. We conducted a user study with 12 participants suggesting that the tool enables people to quickly explore design ideas and experience them immediately. The design suggestions encouraged collaboration, expression, exploration, and enjoyment, which improved creativity support.

Doubly constrained offline reinforcement learning for learning path recommendation

Learning path recommendation refers to the application of interactive recommendation systems in the field of education, aimed at optimizing learning outcomes while minimizing the workload of learners, teachers, and curriculum designers. Reinforcement Learning (RL) has proven effective in capturing and modeling the complex interactions among course activities, learner behaviors, and educational outcomes. Therefore, combining the two approaches presents endless possibilities for personalized education through the use of interactive recommendation systems in the education domain. However, traditional RL algorithms require extensive interaction with the environment during the training phase. Using unverified recommendation logic in interactions with actual students can give rise to unmanageable problems and hinder effective performance in an educational setting. This is because extrapolation introduces substantial evaluation errors that result in recommendations deviating significantly from the actual educational requirements. To address this limitation, we propose a novel method of offline reinforcement learning called Doubly Constrained deep Q-learning Network (DCQN). This method utilizes two generative models to fit existing student historical interaction data, which in turn, constrains the original policy network to generate new actions based on past interactions, avoiding the occurrence of overestimated actions and reducing extrapolation errors. Empirical results on demonstrate that this approach performs better than existing techniques across D4RL, i.e., datasets for deep data-driven reinforcement learning and real educational datasets.

Beyond recommendation acceptance: explanation’s learning effects in a math recommender system

Recommender systems can provide personalized advice on learning for individual students. Providing explanations of those recommendations are expected to increase the transparency and persuasiveness of the system, thus improve students’ adoption of the recommendation. Little research has explored the explanations’ practical effects on learning performance except for the acceptance of recommended learning activities. The recommendation explanations can improve the learning performance if the explanations are designed to contribute to relevant learning skills. This study conducted a comparative experiment (N = 276) in high school classrooms, aiming to investigate whether the use of an explainable math recommender system improves students’ learning performance. We found that the presence of the explanations had positive effects on students’ learning improvement and perceptions of the systems, but not the number of solved quizzes during the learning task. These results imply the possibility that the recommendation explanations may affect students’ meta-cognitive skills and their perceptions, which further contribute to students’ learning improvement. When separating the students based on their prior math abilities, we found a significant correlation between the number of viewed recommendations and the final learning improvement for the students with lower math abilities. This indicates that the students with lower math abilities may benefit from reading their learning progress indicated in the explanations. For students with higher math abilities, their learning improvement was more related to the behavior to select and solve recommended quizzes, which indicates a necessity of more sophisticated and interactive recommender system.

CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System

While personalization increases the utility of recommender systems, it also brings the issue offilter bubbles. e.g., if the system keeps exposing and recommending the items that the user is interested in, it may also make the user feel bored and less satisfied. Existing work studies filter bubbles in static recommendation, where the effect of overexposure is hard to capture. In contrast, we believe it is more meaningful to study the issue in interactive recommendation and optimize long-term user satisfaction. Nevertheless, it is unrealistic to train the model online due to the high cost. As such, we have to leverage offline training data and disentangle the causal effect on user satisfaction.To achieve this goal, we propose a counterfactual interactive recommender system (CIRS) that augments offline reinforcement learning (offline RL) with causal inference. The basic idea is to first learn a causal user model on historical data to capture the overexposure effect of items on user satisfaction. It then uses the learned causal user model to help the planning of the RL policy. To conduct evaluation offline, we innovatively create an authentic RL environment (KuaiEnv) based on a real-world fully observed user rating dataset. The experiments show the effectiveness of CIRS in bursting filter bubbles and achieving long-term success in interactive recommendation. The implementation of CIRS is available via https://github.com/chongminggao/ CIRS-codes.

Efficient Tree Policy with Attention-Based State Representation for Interactive Recommendation

Nowadays, interactive recommendation systems (IRS) play a significant role in our daily life. Recently, reinforcement learning has shown great potential in solving challenging tasks in IRS, since it can focus on long-term profit and can capture the dynamic preference of users. However, existing RL methods for IRS have two typical deficiencies. First, most state representation models use left-to-right recurrent neural networks to capture the user dynamics, which usually fail to handle the long and noisy sequential data in real life. Second, an IRS always needs to handle millions of items, leading to a large discrete action space in RL settings, which has not been fully addressed by the inefficient existing works. To bridge these deficiencies, in this paper, we propose attention-based tree recommendation (ATRec), an efficient tree-structured policy with attention-based state representation for IRS. ATRec uses an attention-based state representation model to effectively capture the user’s dynamic preference hidden in the long and noisy sequence of behaviors. Moreover, to improve the learning efficiency, we propose an efficient tree-structured policy representation method, in which a complete tree is devised to represent the policy, and a novel parameter-sharing strategy is introduced. Extensive experiments are conducted on three real-world datasets and the results show the proposed ATRec obtains 42.3% improvement over some of the state of the arts methods in the hit rate and 21.4% improvement in the mean reciprocal rank of the top 30 ranked items. Additionally, the learning and decision efficiency can also be improved at an average of 35.5%.

Large-Scale Interactive Recommendation With Tree-Structured Reinforcement Learning

Although reinforcement learning (RL) techniques are regarded as promising solutions for interactive recommender systems (IRS), such solutions still face three main challenges, namely, i) time inefficiency when handling large discrete action space in IRS, ii) inability to deal with the cold-start scenarios in IRS, iii) data inefficiency during training the RL-based methods. To tackle these challenges, we propose a generic tree-structured RL framework taking both policy-based and value-based approaches into consideration. We propose to construct a balanced tree over representations of the items, such that picking an item is formulated as seeking a suitable path from the root to a leaf node in the balanced tree, which dramatically reduces the time complexity of item recommendation. Further, for cold-start scenarios where prior information of the items is unavailable, we initialize a random balanced tree as the starting point and then refine the tree structure based on the learned item representations. Besides, we also incorporate a user modeling component to explicitly model the environment, which can be utilized in the training phase to improve data efficiency. Extensive experiments on two real-world datasets are conducted and demonstrate that our framework can achieve superior recommendation performance and provide time and data efficiency improvement over state-of-the-art methods in both warm-start and cold-start IRS scenarios.

DIRS-KG: a KG-enhanced interactive recommender system based on deep reinforcement learning

DIRS-KG: a KG-enhanced interactive recommender system based on deep reinforcement learning

REDRL: A review-enhanced Deep Reinforcement Learning model for interactive recommendation

Recent advances in interactive recommender systems (IRS) have received wide attention due to its flexible recommendation strategy and optimization for users’ long-term utility. Considering this interaction paradigm of IRS, researchers have made some attempts to incorporate reinforcement learning (RL) models into IRS, because of the excellent ability of RL in long-term optimizing and decision making. However, data sparsity is an intractable problem most IRS urgently need to address. Although a small amount of work has exploited reviews to address data sparsity, they ignored the varying importance of items for modeling the user. In addition, most existing RL-based approaches suffer from decision-making difficulties when the action space becomes large. To solve above problems, in this work, we present a Review-enhanced Deep Reinforcement Learning model (REDRL) for interactive recommendation. Specifically, we utilize text reviews, combined with a pretrained review representation model to acquire item review-enhanced embedding representations. Then we formalize the recommendation problem as a Markov Decision Process (MDP), and exploit deep reinforcement learning (DRL) to model the interactive recommendation. Notably, we introduce a multi-head self-attention technique to capture distinct importance of various items in the sequence behavior, which is overlooked by existing work when modeling the user preference. In this way, we can model long-term dynamic preferences of users accurately and discriminately for comprehensive interactive recommendation. Moreover, we subtly combine the semantic structure information in the user–item bipartite graph with meta-paths in heterogeneous information networks (HIN), to filter some irrelevant items and obtain candidate items dynamically. By this means, the size of the discrete action space is effectively reduced from a new anger. The experimental results based on three benchmark datasets demonstrate the efficiency of our method with significant improvement over state-of-the-art.

An Interactive Personalized Garment Design Recommendation System Using Intelligent Techniques

This paper presents a garment design recommendation system based on two mathematical models that permit the prediction and control of garment styles and structural parameters from a consumer’s personalized requirements in terms of fitting and aesthetics. Based on a formalized professional garment knowledge base, enabling the quantitative characterization of the relations between consumer profiles and garment profiles (colors, fabrics, styles, and garment fit), these two models aim at recommending the most relevant garment profile from a specific consumer profile, using reasoning with fuzzy rules and self-adjusting the garment patterns according to the feedback of the 3D virtual fitting effects corresponding to the recommended garment profile, using a genetic algorithm (GA) and support vector regression. Based on these knowledge-based models, the proposed interactive recommendation system enables the progressive optimization of the design solution through a series of human–machine interactions, i.e., the repeated execution of the cycle “design generation—virtual garment demonstration—user’s evaluation—adjustment” until the satisfaction of the end user (consumer or designer). The effectiveness of this interactive recommendation system was validated by a real case of pants customization. In a manner different from the existing approaches, the proposed system will enable designers to rapidly, accurately, intelligently, and automatically generate the optimal design solution, which is relevant in dealing with mass customization and e-shopping for fashion companies.

Towards Increasing the Coverage of Interactive Recommendations

An interactive recommender system pursues two somewhat contradictory goals. On one hand, the system should provide highly relevant recommendations with the best match to the overall user needs. On the other hand, the recommendations should be sufficiently diverse to cover a range of users' possible interests. Such recommendations increase chances that the user finds items that match their context while also informing the system which items are currently most important. In this paper, we present a ranking approach that balances the demands of relevance and coverage. We evaluate the approach on two problems, of advisor and movie recommendations, where the immediate needs of the user are likely to be diverse. Our approach considerably increases chances that the user finds relevant items in the first few steps of the recommendation dialog.

Adversarial Robustness of Deep Reinforcement Learning Based Dynamic Recommender Systems.

Adversarial attacks, e.g., adversarial perturbations of the input and adversarial samples, pose significant challenges to machine learning and deep learning techniques, including interactive recommendation systems. The latent embedding space of those techniques makes adversarial attacks challenging to detect at an early stage. Recent advance in causality shows that counterfactual can also be considered one of the ways to generate the adversarial samples drawn from different distribution as the training samples. We propose to explore adversarial examples and attack agnostic detection on reinforcement learning (RL)-based interactive recommendation systems. We first craft different types of adversarial examples by adding perturbations to the input and intervening on the casual factors. Then, we augment recommendation systems by detecting potential attacks with a deep learning-based classifier based on the crafted data. Finally, we study the attack strength and frequency of adversarial examples and evaluate our model on standard datasets with multiple crafting methods. Our extensive experiments show that most adversarial attacks are effective, and both attack strength and attack frequency impact the attack performance. The strategically-timed attack achieves comparative attack performance with only 1/3 to 1/2 attack frequency. Besides, our white-box detector trained with one crafting method has the generalization ability over several other crafting methods.

Production Assessment Using a Knowledge Transfer Framework and Evidence Theory

Operational knowledge is one of the most valuable assets in a company, as it provides a strategic advantage over competitors and ensures steady and optimal operation in machines. An (interactive) assessment system on the shop floor can optimize the process and reduce stopovers because it can provide constant valuable information regarding the machine condition to the operators. However, formalizing operational (tacit) knowledge to explicit knowledge is not an easy task. This transformation considers modeling expert knowledge, quantification of knowledge uncertainty, and validation of the acquired knowledge. This study proposes a novel approach for production assessment using a knowledge transfer framework and evidence theory to address the aforementioned challenges. The main contribution of this paper is a methodology for the formalization of tacit knowledge based on an extended failure mode and effect analysis for knowledge extraction, as well as the use of evidence theory for the uncertainty definition of knowledge. Moreover, this approach uses primitive recursive functions for knowledge modeling and proposes a validation strategy of the knowledge using machine data. These elements are integrated into an interactive recommendation system hosted on a backend that uses HoloLens as a visual interface. We demonstrate this approach using an industrial setup: a laboratory bulk good system. The results yield interesting insights, including the knowledge validation, uncertainty behavior of knowledge, and interactive troubleshooting for the machine operator.

Semantic Trajectory Analytics and Recommender Systems in Cultural Spaces

Semantic trajectory analytics and personalised recommender systems that enhance user experience are modern research topics that are increasingly getting attention. Semantic trajectories can efficiently model human movement for further analysis and pattern recognition, while personalised recommender systems can adapt to constantly changing user needs and provide meaningful and optimised suggestions. This paper focuses on the investigation of open issues and challenges at the intersection of these two topics, emphasising semantic technologies and machine learning techniques. The goal of this paper is twofold: (a) to critically review related work on semantic trajectories and knowledge-based interactive recommender systems, and (b) to propose a high-level framework, by describing its requirements. The paper presents a system architecture design for the recognition of semantic trajectory patterns and for the inferencing of possible synthesis of visitor trajectories in cultural spaces, such as museums, making suggestions for new trajectories that optimise cultural experiences.