Sequence modelling for e-commerce

Abstract

With the proliferation of electronic commerce (e-commerce), the data generated by both customers and service providers can accumulate at a fast rate. As such, analyzing the rich but subtle patterns within the e-commerce data offers a prominent opportunity of refining user experience and increasing business revenue. Due to the high velocity of e-commerce data, sequence modelling plays a pivotal role in delivering timely predictive analytics and recommendations. Based on the granularity of data, sequence modelling for e-commerce is mainly conducted at two levels, namely macro-level modelling and micro-level modelling. When researching on e-commerce data, macro-level sequence modelling aims to understand the evolution of high-level business trends in order to set the foundation for enterprise strategic planning, e.g., sales prediction for inventory management. Meanwhile, micro-level sequence modelling focuses on learning fine-grained and dynamic user preferences from behavioral data to deliver personalized user experience, e.g., recommendation systems deployed by all major e-commerce platforms. In our research, we aim to effectively tackle sequence modelling in e-commerce scenarios at different levels, and then propose a unified model that allows for both macro- and micro-level sequence modelling, thus supporting a wide range of e-commerce applications. In summary, our research consists of the following three parts.Firstly, for macro-level sequence modelling, we solve the problem of sales prediction, which is a critical means to achieve a healthy balance between supply and demand in e-commerce. The sales prediction task is formulated as a time series prediction problem which aims to predict the future sales volume for different products with observed influential factors (e.g., brand, season, discount, etc.) and corresponding historical sales records. However, with the development of contemporary commercial markets, the dynamic interactions between influential factors with different semantic meanings become more subtle, causing challenges in fully capturing dependencies among these variables. Besides, though seeking similar trends from the history benefits the accuracy for the prediction, existing methods hardly suit sales prediction tasks because the trends in sales data are more irregular and complex. Hence, we gain insights from the encoder-decoder recurrent neural network (RNN) structure, and propose a novel framework named TADA to carry out trend alignment with dual-attention, multi-task RNNs for sales prediction. In TADA, we innovatively divide the influential factors into internal feature and external feature, which are jointly modelled by a multi-task RNN encoder. In the decoding stage, TADA utilizes two attention mechanisms to compensate for the unknown states of influential factors in the future and adaptively align the upcoming trend with relevant historical trends to ensure precise sales prediction.nSecondly, for micro-level sequence modelling, we investigate sequential top-k recommendation, which infers users' preferences from their sequential behaviors and predicts their next interested items. Though it is important to capture the sequential patterns from the user-item interaction data, existing methods only focus on modelling the sparse item-wise sequential effect in user preference and only consider the homogeneous user interaction behaviors (i.e., a single type of user behavior). As a result, the data sparsity issue inevitably arises and makes the learned sequential patterns fragile and unreliable, impeding the sequential recommendation performance of existing methods. Hence, in this task, we propose AIR, namely attentional intention-aware recommender systems to predict category-wise future user intention and collectively exploit the rich heterogeneous user interaction behaviors (i.e., multiple types of user behaviors). In AIR, we propose to represent user intention as an action-category tuple to discover category-wise sequential patterns and to capture varied effect of different types of actions for recommendation. A novel attentional recurrent neural network (ARNN) is proposed to model the intention migration effect and infer users' future intention. Besides, an intention-aware factorization machine (ITFM) is developed to perform intention-aware sequential recommendation.Lastly, we develop a machine learning model that is generalizable to both macro- and micro-level sequence modelling tasks in e-commerce. Specifically, we extend a versatile predictive model, namely factorization machines (FMs) to the sequential setting. In e-commerce, models based on FMs are capable of modelling high-order interactions among features for effective predictive analytics, e.g., targeted advertising and recommendation. However, existing FM-based models assume no temporal orders in the data, and are unable to capture the sequential dependencies or patterns within the dynamic features, impeding the performance and adaptivity of these methods. Hence, we propose a novel sequence-aware factorization machine (SeqFM) for sequential predictive analytics, which models feature interactions by fully investigating the effect of sequential dependencies. As static features (e.g., user gender) and dynamic features (e.g., users' interacted items) express different semantics, we innovatively devise a multi-view self-attention scheme that separately models the effect of static features, dynamic features and the mutual interactions between static and dynamic features in three different views. In SeqFM, we further map the learned representations of feature interactions to the desired output with a shared residual network.n