Next-item Recommendation with Bidirectional Encoder Representations from Transformer and Matrix Factorization

Abstract

The next-item recommendation is a task based on sequential nature of users' behaviors. Existing approaches employ sequential neural networks with users' historical interactions for predicting next-item. However, these methods have the limitations including: (a) traditional unidirectional architectures overemp-hasize a rigidly order between items among sequence which is not always necessarily; (b) they often take users' short-term preferences from users' transient interests into consideration while neglect users' long-term intentions. In fact, consumers' purchasing intensions are not only affected by their recent preferences, but also by their long-term preferences for certain items. To this end, we propose a sequential recommendation model for next-item called BERTMF, which takes both users' recent intentions and long-term preferences into account for next-item recommendation. For users' current intentions, we employ bidirectional encoder representations from transformer (BERT) to model users' behavior sequences. To address users' diverse behavioral intentions tendency, attention mechanism is embedded into users' short-term intentions representations. In this work, we propose a generalized matrix factorization (GMF) method to model users' long-term interests and target it to the attention mechanism. As users' current intentions are changing and relative to users and target item, we introduce a fusion layer to better model both users' sequential behaviors and long-term intention representations. In the experiments on public datasets, our approach outperforms various origin next-item recommendation systems.