Exploring Clustering-Based Reinforcement Learning for Personalized Book Recommendation in Digital Library

Xinhua Wang,Yuchen Wang,Baozhong Gao,Lei Guo,Liancheng Xu,Fangai Liu,Wei Li

doi:10.3390/info12050198

Abstract

Digital library as one of the most important ways in helping students acquire professional knowledge and improve their professional level has gained great attention in recent years. However, its large collection (especially the book resources) hinders students from finding the resources that they are interested in. To overcome this challenge, many researchers have already turned to recommendation algorithms. Compared with traditional recommendation tasks, in the digital library, there are two challenges in book recommendation problems. The first is that users may borrow books that they are not interested in (i.e., noisy borrowing behaviours), such as borrowing books for classmates. The second is that the number of books in a digital library is usually very large, which means one student can only borrow a small set of books in history (i.e., data sparsity issue). As the noisy interactions in students’ borrowing sequences may harm the recommendation performance of a book recommender, we focus on refining recommendations via filtering out data noises. Moreover, due to the the lack of direct supervision information, we treat noise filtering in sequences as a decision-making process and innovatively introduce a reinforcement learning method as our recommendation framework. Furthermore, to overcome the sparsity issue of students’ borrowing behaviours, a clustering-based reinforcement learning algorithm is further developed. Experimental results on two real-world datasets demonstrate the superiority of our proposed method compared with several state-of-the-art recommendation methods.

Highlights

Digital library as one of the most important ways in helping students acquire professional knowledge and improve their professional level has gained great attention recently
The main contributions of this work are listed as follows: 1. We introduce HRL into the book recommendation task in the digital library, where a basic book recommender is first pre-trained, and a hierarchical agent is devised to filter out the interactions that might miss leading this recommender
To solve the data noisy and data sparseness challenges in book recommendation, we propose a Clustering-based Hierarchical Reinforcement Learning Network (CHRL) as our solution, whose main idea is to leverage the power of the clustering-based reinforcement learning technique to filter out noisy interactions that may mislead the recommendation algorithms

Summary

Introduction

Digital library as one of the most important ways in helping students acquire professional knowledge and improve their professional level has gained great attention recently. It is a huge challenge for students to find the required resources (such as books, reports, and periodicals). To overcome this challenge, we resort to recommender systems [1,2,3,4], which can leverage users’ historical records to help them efficiently discover interesting and high-quality information. The book recommendation task in a digital library is to recommend books at time t + 1 to a set of users given the historical book borrowing records before time t

Methods

Results

Conclusion