Efficient Tree Policy with Attention-Based State Representation for Interactive Recommendation

Abstract

Nowadays, interactive recommendation systems (IRS) play a significant role in our daily life. Recently, reinforcement learning has shown great potential in solving challenging tasks in IRS, since it can focus on long-term profit and can capture the dynamic preference of users. However, existing RL methods for IRS have two typical deficiencies. First, most state representation models use left-to-right recurrent neural networks to capture the user dynamics, which usually fail to handle the long and noisy sequential data in real life. Second, an IRS always needs to handle millions of items, leading to a large discrete action space in RL settings, which has not been fully addressed by the inefficient existing works. To bridge these deficiencies, in this paper, we propose attention-based tree recommendation (ATRec), an efficient tree-structured policy with attention-based state representation for IRS. ATRec uses an attention-based state representation model to effectively capture the user’s dynamic preference hidden in the long and noisy sequence of behaviors. Moreover, to improve the learning efficiency, we propose an efficient tree-structured policy representation method, in which a complete tree is devised to represent the policy, and a novel parameter-sharing strategy is introduced. Extensive experiments are conducted on three real-world datasets and the results show the proposed ATRec obtains 42.3% improvement over some of the state of the arts methods in the hit rate and 21.4% improvement in the mean reciprocal rank of the top 30 ranked items. Additionally, the learning and decision efficiency can also be improved at an average of 35.5%.