Long- and short-term self-attention network for sequential recommendation

Abstract

With great value in real applications, sequential recommendation aims to recommend users the personalized sequential actions. To achieve better performance, it is essential to consider both long-term preferences and sequential patterns (i.e., short-term dynamics). Compared to widely used Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN), Self-Attention Network (SAN) obtains a surge of interest due to fewer parameters, highly parallelizable computation, and flexibility in modeling dependencies. However, existing SAN-based models are inadequate in characterizing and distinguishing users’ long-term preferences and short-term demands since they do not emphasize the importance of the current interest and temporal order information of sequences. In this paper, we propose a novel multi-layer long- and short-term self-attention network (LSSA) for sequential recommendation. Specifically, we first split the entire sequence of a user into multiple sub-sequences according to the timespan. Then the first self-attention layer learns the user’s short-term dynamics based on the last sub-sequence, while the second one captures the user’s long-term preferences through the previous sub-sequences and the last one. Finally, we integrate the long- and short-term representations together to form the user’s final hybrid representation. We evaluate the proposed model on three real-world datasets, and our experimental results show that LSSA outperforms state-of-the-art methods with a wide margin.