Hierarchical reinforcement learning with dynamic recurrent mechanism for course recommendation

Abstract

In online learning scenarios, the learners usually hope to find courses that meet their preferences and the needs for their future developments. Thus, there is a great need to develop effective personalized course recommender systems that can guide the learners to choose suitable courses. In practice, Reinforcement Learning (RL) can be applied to build dynamic user profiles from users’ interactions with courses, which is the key to the success of a course recommender system. However, existing RL-based course recommendation methods usually suffer from the trade-off between exploration and exploitation. In this paper, we propose a novel course recommendation model, namely Hierarchical rEinforcement Learning with dynAmic Recurrent mechanism (HELAR), in which a profile constructor with autonomous learning ability is designed to make personalized course recommendation. To address the exploration-exploitation trade-off issue in constructing user profiles, we propose a novel policy gradient method. It employs a recurrent scheme by context-aware learning to exploit the current knowledge, while utilizing a dynamic baseline to explore the user’s future preferences. Extensive experiments are conducted on two real-world datasets to evaluate the performance of the proposed HELAR model, and the experimental results demonstrate the advantage of HELAR over state-of-the-art course recommendation methods.