UET4Rec: U-net encapsulated transformer for sequential recommender

Abstract

Recommending a tempting sequence of items according to a user’s previous history of purchases and clicks, for instance, in the online shopping portals is challenging. And yet it is a crucial task for all service providers. One of the core components in the recommender systems is a sequential model with which an input sequence is transformed into the predicted items. Among many, deep neural networks, such as RNN, LSTM, and Transformer, have been favored for this purpose. However, improving the performance of these models remains an important task. To address this, we propose a novel sequential model by combining a U-net and Transformer, called U-net Encapsulated Transformer. This hybrid architecture places a transformer model between a convolutional encoder and its decoder, wherein each convolutional layer processes 1D signal, i.e. text. The primary benefit of this structure is that the computational burden is reduced since the embedding size of the input to the Transformer is drastically decreased as the signal has to go through the multi-layer convolutional encoder. This solution leverages recommendation lists for action predictions and uses user feedback as direct rewards for updating the model. In addition, the loss function mechanism is improved by including contrastive and reinforcement learning losses. Evaluation of the proposed model, including extensive ablation study, is carried out on four standard benchmark datasets, such as RC15, RetailRocket, MovieLens-1M, and Amazon-Beauty, demonstrating superior performance compared to state-of-the-art methods.