Accurate multi-interest modeling for sequential recommendation with attention and distillation capsule network

Abstract

The objective of sequential recommendation systems is to assist users in efficiently discovering relevant information that they need within vast online environments. Existing studies have frequently overlooked users’ diverse interests, failing to capture their multiple latent interests effectively, particularly in cases wherein users have extensive interaction histories. We introduce a novel approach, called Accurate Multi-Interest Modeling for sequential recommendation to address this limitation. The proposed model combines users’ long- and short-term preferences by using an attention-based module, allowing for comprehensive preference modeling. Furthermore, it incorporates a distillation capsule network-based module that models different user interests, with each capsule representing a unique interest. These interests are further refined using an interest distiller to identify the top-k interests of each user. Moreover, a module based on the multi-head self-attention mechanism captures hidden preference transition patterns within user interaction sequences. Finally, a complete user interest representation is obtained by aggregating the outputs of the aforementioned modules. This decoupling-then-fusion strategy allows the model to capture various detailed features better and consider the interactions among different features to model user needs more comprehensively. Experimental results conducted on three datasets unequivocally demonstrate the superiority of our proposed model.