An Attention-Based Latent Information Extraction Network (ALIEN) for High-Order Feature Interactions

Ruo Huang,Zhonghong Ou,Meina Song,Haihong E,Shelby Mcintyre

doi:10.3390/app10165468

Abstract

One of the primary tasks for commercial recommender systems is to predict the probabilities of users clicking items, e.g., advertisements, music and products. This is because such predictions have a decisive impact on profitability. The classic recommendation algorithm, collaborative filtering (CF), still plays a vital role in many industrial recommender systems. However, although straight CF is good at capturing similar users’ preferences for items based on their past interactions, it lacks regarding (1) modeling the influences of users’ sequential patterns from their individual history interaction sequences and (2) the relevance of users’ and items’ attributes. In this work, we developed an attention-based latent information extraction network (ALIEN) for click-through rate prediction, to integrate (1) implicit user similarity in terms of click patterns (analogous to CF), and (2) modeling the low and high-order feature interactions and (3) historical sequence information. The new model is based on the deep learning, which goes beyond the capabilities of econometric approaches, such as matrix factorization (MF) and k-means. In addition, the approach provides explainability to the recommendation by interpreting the contributions of different features and historical interactions. We have conducted experiments on real-world datasets that demonstrate considerable improvements over strong baselines.

Highlights

In recent years, recommender systems have improved substantially and are widely adopted by many online services in domains such as news, e-commerce and social media, among many others.The key to a personalized recommendation for a target user is in modeling similar users’ preferences for items in a domain based on those similar users’ past interactions and the similarity of their patterns to those of the target user
Our goal is to make precise the click-through rate (CTR) prediction by modeling u and vc ’s low and high-order feature interactions and u’s history interactions. u’s history interactions consist of u’s history items before he or she interacts with vc, and the embedding set of u’s history items is denoted as Suvc = {v1, v2, ..., v|Suvc | }
To precisely and comprehensively capture a user’s diverse interests, similar to the activation unit applied in [8], we propose the user behavior-driven latent information extraction layer (HISTORY) to learn the latent information from user’s historical behavior by modeling the user’s history interactions, and investigating the different impact which each history item vi ∈ Suvc has on the candidate item vc

Summary

Introduction

The key to a personalized recommendation for a target user is in modeling similar users’ preferences for items in a domain based on those similar users’ past interactions and the similarity of their patterns to those of the target user (e.g., in ratings and clicks). The most well-known collaborative filtering technique, matrix factorization [3,4,5], projects users and items into a shared latent space and utilizes a vector of latent dimensions to represent a user or an item. Thereafter a user’s interaction with an item is modeled as the inner product of their latent vectors. Researchers have been embracing deep-learning neural architectures that can learn very complicated functions from data, to replace the inner product applied in matrix factorization [6,7] and include information from history sequences [8,9]

Objectives

Methods

Results

Conclusion