Learning Style Integrated Deep Reinforcement Learning Framework for Programming Problem Recommendation in Online Judge System

Abstract

Exercise recommendation is an integral part of enabling personalized learning. Giving appropriate exercises can facilitate learning for learners. The programming problem recommendation is a specific application of the exercise recommendation. Therefore, an innovative recommendation framework for programming problems that integrate learners’ learning styles is proposed. In addition, there are some difficulties to be solved in this framework, such as quantifying learning behavior, representing programming problems, and quantifying learning strategies. For the difficulties in quantifying learning behavior and quantifying learning strategies, a programming problem recommendation algorithm based on deep reinforcement learning (DRLP) is proposed. DRLP includes the specific design of action space, action-value Q-network, and reward function. Learning style is embedded into DRLP through action space to make recommendations more personalized. To represent the programming problem in DRLP, a multi-dimensional integrated programming problem representation model is proposed to quantify the difficulty feature, knowledge point feature, text description, input description, and output description of programming problems. In particular, Bi-GRU is introduced to learn texts’ contextual semantic association information from both positive and negative directions. Finally, a simulation experiment is carried out with the actual learning behavior data of 47,147 learners in the LUOGU Online Judge system. Compared with the optimal baseline model, the recommendation effect of DRLP has improved (HR, MRR, and Novelty have increased by 4.35%, 1.15%, and 1.1%), which proves the rationality of the programming problem representation model and action-value Q-network.