Large-scale Recommender Systems Research Articles

Ada-Retrieval: An Adaptive Multi-Round Retrieval Paradigm for Sequential Recommendations

Retrieval models aim at selecting a small set of item candidates which match the preference of a given user. They play a vital role in large-scale recommender systems since subsequent models such as rankers highly depend on the quality of item candidates. However, most existing retrieval models employ a single-round inference paradigm, which may not adequately capture the dynamic nature of user preferences and stuck in one area in the item space. In this paper, we propose Ada-Retrieval, an adaptive multi-round retrieval paradigm for recommender systems that iteratively refines user representations to better capture potential candidates in the full item space. Ada-Retrieval comprises two key modules: the item representation adapter and the user representation adapter, designed to inject context information into items' and users' representations. The framework maintains a model-agnostic design, allowing seamless integration with various backbone models such as RNNs or Transformers. We perform experiments on three widely used public datasets, incorporating five powerful sequential recommenders as backbone models. Our results demonstrate that Ada-Retrieval significantly enhances the performance of various base models, with consistent improvements observed across different datasets. Our code and data are publicly available at: https://github.com/ll0ruc/Ada-Retrieval.

Enhancing User Interest based on Stream Clustering and Memory Networks in Large-Scale Recommender Systems

Enhancing User Interest based on Stream Clustering and Memory Networks in Large-Scale Recommender Systems

An Off-Policy Reinforcement Learning Algorithm Customized for Multi-Task Fusion in Large-Scale Recommender Systems

An Off-Policy Reinforcement Learning Algorithm Customized for Multi-Task Fusion in Large-Scale Recommender Systems

Non-Stationary Linear Bandits With Dimensionality Reduction for Large-Scale Recommender Systems

Non-Stationary Linear Bandits With Dimensionality Reduction for Large-Scale Recommender Systems

Enhancing Image Recommendations in Social Contexts using Hierarchical Attention

The paper introduces a novel approach to address the computational challenges faced by recommender systems operating on large-scale datasets, particularly in the context of social contextual image recommendation. Recognizing the need for a more efficient means of comparing numerous items to identify users' preferences, the proposed hierarchical consideration model delves into three crucial factors: transfer history, social impact, and owner adoration. These factors encapsulate nuanced aspects of user preferences, deriving from intricate relationships between users and images. To operationalize this, a hierarchical attention network is designed, explicitly reflecting the hierarchical nature of users' latent interests within the identified key aspects. Leveraging embeddings from state-of-the-art deep learning models tailored for different data types, the hierarchical attention network dynamically adjusts its focus on varying content levels. Extensive experimentation on real-world datasets underscores the model's superiority, with compelling results demonstrating its effectiveness and adaptability, particularly in contrast to existing approaches. The study culminates in highlighting the model's prowess in navigating diverse data landscapes, establishing its potential as an advanced solution for large-scale recommender systems.

Contrastive representation learning in recommendation systems--The investigation of the performance of the self-supervised learning in large-scale recommendation systems

Self-supervised learning (SSL) has been proposed in machine learning projects for its convenience of reducing self-labeled datasets in recent years. However, the implementation of SSL in large-scale recommendation systems has lagged behind the evolution because of their scarce and tailed characteristics. In 2021, an article proposed the use of SSL in recommendation systems to pursue an improvement in the performance of recommender models. This article is built on this previous investigation and aims to further explore the role of SSL in recommendation systems and to investigate an improvement of the models efficiency. To answer research questions, this paper tests three models with different numbers of towers to discover the best performance of the use of SSL in recommender models. Consequently, it is found that implementing SSL on the item side only (two-tower DNNs) produced the best result. Then, when constructing the two-tower DNNs model, this article examines different numbers of negative pairs to change the InfoNCE loss to investigate a tradeoff between the number of positive and negative samples in the performance of the model. As a result, it turns out to be a weak correlation between this ratio and the performance; Hence, it is concluded that the change of the number of positive and negative samples would not necessarily affect the two-tower DNNs model. In our experiential stage, this paper uses a real-world dataset with 100k training samples to testify and compare our results.

LightFR: Lightweight Federated Recommendation with Privacy-preserving Matrix Factorization

Federated recommender system (FRS), which enables many local devices to train a shared model jointly without transmitting local raw data, has become a prevalent recommendation paradigm with privacy-preserving advantages. However, previous work on FRS performs similarity search via inner product in continuous embedding space, which causes an efficiency bottleneck when the scale of items is extremely large. We argue that such a scheme in federated settings ignores the limited capacities in resource-constrained user devices ( i.e. , storage space, computational overhead, and communication bandwidth), and makes it harder to be deployed in large-scale recommender systems. Besides, it has been shown that transmitting local gradients in real-valued form between server and clients may leak users’ private information. To this end, we propose a lightweight federated recommendation framework with privacy-preserving matrix factorization, LightFR , that is able to generate high-quality binary codes by exploiting learning to hash technique under federated settings, and thus enjoys both fast online inference and economic memory consumption. Moreover, we devise an efficient federated discrete optimization algorithm to collaboratively train model parameters between the server and clients, which can effectively prevent real-valued gradient attacks from malicious parties. Through extensive experiments on four real-world datasets, we show that our LightFR model outperforms several state-of-the-art FRS methods in terms of recommendation accuracy, inference efficiency and data privacy.

Improving Accuracy and Diversity in Matching of Recommendation With Diversified Preference Network

Real-world recommendation systems need to deal with millions of item candidates. Therefore, most practical large-scale recommendation systems usually contain two modules. The matching module aims to efficiently retrieve hundreds of high-quality items from large corpora, while the ranking module aims to generate specific ranks for these items. Recommendation diversity is an essential factor that strongly impacts user experience. There are lots of efforts that have explored recommendation diversity in ranking, while the matching module should take more responsibility for diversity. In this article, we propose a novel Heterogeneous graph neural network framework for diversified recommendation (GraphDR) in matching to improve both recommendation accuracy and diversity. Specifically, GraphDR builds a huge heterogeneous preference network to record different types of user preferences, and conducts a field-level heterogeneous graph attention network for node aggregation. We conduct a neighbor-similarity based loss with a multi-channel matching to improve both accuracy and diversity. In experiments, we conduct extensive online and offline evaluations on a real-world recommendation system with various accuracy and diversity metrics and achieve significant improvements. GraphDR has been deployed on a well-known recommendation system named WeChat Top Stories, which affects millions of users. The source code will be released in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/lqfarmer/GraphDR</uri> .

A differentially private matrix factorization based on vector perturbation for recommender system

Matrix factorization (MF) techniques have yielded immense success in recommender systems (RSs). Since a huge amount of user data is collected and used in RS, it raises concerns about data privacy. As a strict privacy protection framework, many efforts attempt to apply Differential Privacy (DP) to MF. However, there are still some challenges or problems in designing MF with privacy preservation, such as error accumulation in the multiple iterations of MF, introduction of unnecessary noises, and difficult sensitivity analysis. To overcome these problems, we devise a vector perturbation-based differentially private matrix factorization (VP-DPMF). Our scheme can prevent error accumulation by perturbing the objective function of MF rather than its factorization process or results. It also addresses the difficulty of analyzing sensitivity by exploiting the polynomial representation of the objective function. Furthermore, our scheme can reduce unnecessary noises by controlling the perturbation within the vector term of the polynomial, and can preserve the convexity property of the original function. Theoretical analysis demonstrates that our scheme can achieve good performance in a large-scale recommender system. Experimental results on some benchmark datasets show that the proposed scheme can provide both rigid privacy guarantee and satisfactory recommendation quality.

HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation

An efficient solution to the large-scale recommender system is to represent users and items as binary hash codes in the Hamming space. Towards this end, existing methods tend to code users by modeling their Hamming similarities with the items they historically interact with, which are termed as the first-order similarities in this work. Despite their efficiency, these methods suffer from the suboptimal representative capacity, since they forgo the correlation established by connecting multiple first-order similarities, i.e., the relation among the indirect instances, which could be defined as the high-order similarity. To tackle this drawback, we propose to model both the first- and the high-order similarities in the Hamming space through the user-item bipartite graph. Therefore, we develop a novel learning to hash framework, namely Hamming Spatial Graph Convolutional Networks (HS-GCN), which explicitly models the Hamming similarity and embeds it into the codes of users and items. Extensive experiments on three public benchmark datasets demonstrate that our proposed model significantly outperforms several state-of-the-art hashing models, and obtains performance comparable with the real-valued recommendation models.

Android malware detection via an app similarity graph

Due to the ever-increasing number of Android applications and constant advances in software development techniques, there is a need for scalable and flexible malware detectors that can efficiently address big data challenges. Motivated by large-scale recommender systems, we propose a static Android application analysis method which relies on an app similarity graph (ASG). We believe that the key to classifying app’s behavior lies in their common reusable building blocks, e.g. functions, in contrast to expert based features. We demonstrate our method on the Drebin benchmark in both balanced and unbalanced settings, on a brand new VTAz dataset from 2020, and on a dataset of approximately 190K applications provided by VirusTotal, achieving an accuracy of 0.975 in balanced settings, and AUC score of 0.987. The analysis and classification time of the proposed methods are notably lower than in the reviewed research (from 0.08 to 0.153 sec/app).

Learning over No-Preferred and Preferred Sequence of Items for Robust Recommendation

In this paper, we propose a theoretically supported sequential strategy for training a large-scale Recommender System (RS) over implicit feedback, mainly in the form of clicks. The proposed approach consists in minimizing pairwise ranking loss over blocks of consecutive items constituted by a sequence of non-clicked items followed by a clicked one for each user. We present two variants of this strategy where model parameters are updated using either the momentum method or a gradient-based approach. To prevent updating the parameters for an abnormally high number of clicks over some targeted items (mainly due to bots), we introduce an upper and a lower threshold on the number of updates for each user. These thresholds are estimated over the distribution of the number of blocks in the training set. They affect the decision of RS by shifting the distribution of items that are shown to the users. Furthermore, we provide a convergence analysis of both algorithms and demonstrate their practical efficiency over six large-scale collections with respect to various ranking measures and computational time.

Multi-modal Discrete Collaborative Filtering for Efficient Cold-start Recommendation

Hashing is an effective technique to improve the efficiency of large-scale recommender system by representing both users and items into binary codes. However, existing hashing-based recommendation methods still suffer from two important problems: 1) Cold-start. They employ the user-item interactions and single auxiliary information to learn the binary hash codes. But the full interaction history is not always available and the single auxiliary information may be missing. 2) Efficient optimization. They learn the hash codes with two-step relaxed optimization or one-step discrete hash optimization based on the cyclic coordinate descent, which results in significant quantization loss or still consumes considerable computation time. In this paper, we propose a Multi-modal Discrete Collaborative Filtering (MDCF) for efficient cold-start recommendation. We map the multi-modal features of users and items to a consensus Hamming space based on the matrix factorization framework. Specifically, a low-rank self-weighted multi-modal fusion module is designed to adaptively fuse the multi-modal features into binary hash codes. Additionally, to support large-scale recommendation, a fast discrete optimization method based on augmented Lagrangian multiplier is developed to directly compute the binary hash codes with simple operations. Experiments show the superior performance of the proposed method over state-of-the-art baselines.

Deep Multi-Objective Multi-Stakeholder Music Recommendation

Many successful recommendation approaches rely on the optimization of a single objective function and focus on predicting ratings by considering customer and product features. In this paper, we consider the multi-objective recommendation problem for several stakeholders and introduce a large-scale recommender system that aims at satisfying multiple, potentially conflicting, objectives of consumers and vendors. We propose a two-tower neural network model for music recommendation that: (a) learns each of the stakeholders' objectives in a different tower, (b) shares the latent information that was learned in each tower to predict each stakeholder's objective, and (c) aggregates the predicted objectives to generate rating-based recommendations. Specifically, we focus on the following criteria: user satisfaction objectives, such as saves, likes, and degrees of engagement with songs; and artist satisfaction objectives, such as acquiring new fans. We apply our proposed deep architecture to music recommendation and examine the performance of our model on two proprietary industrial-scale datasets provided by Spotify and on a public music listening history dataset from Last.fm. We find that our model is effective in solving the multi-objective problem and achieves the best performance trade-off for the two stakeholders (users and artists), when compared to the original single-objective model and state-of-the-art multi-objective and multi-stakeholder methods.

Large‐Scale Personalized Categorization of Financial Transactions

A major part of financial accounting involves organizing business transactions using a customizable filing system that accountants call a “chart of accounts.” This task must be carried out for every financial transaction, and hence automation is of significant value to the users of accounting software. In this article we present a large‐scale recommendation system used by millions of small businesses in the USA, UK, Australia, Canada, India, and France to organize billions of financial transactions each year. The system uses machine learning to combine fragments of information from millions of users in a manner that allows us to accurately recommend chart‐of‐accounts categories even when users have created their own or named them using abbreviations or in foreign languages. Transactions are handled even if a given user has never categorized a transaction like that before. The development of such a system and testing it at scale over billions of transactions is a first in the financial industry.

Enhancing the Role of Large-Scale Recommendation Systems in the IoT Context

The Internet of Things (IoT) connects heterogeneous physical devices with the ability to collect data using sensors and actuators. These data can infer useful information for decision-makers in many applications systems monitoring, healthcare, transportations, data storage, smart homes, and many others. In the Era of the Internet of Things (IoT), recommender systems can support scenarios such as recommending apps, IoT workflows, services, sensor equipment, hotels, and drugs to users and customers. Current state-of-art recommendation systems, including collaborative filtering methods, suffer from scalability and sparsity problems. This article proposes a clustering-based recommendation system that adopts the vector space model from information retrieval to obtain highly accurate recommendations. The proposed algorithm uses four well-known clustering techniques as k-means (KM), fuzzy c-mean (FCM), Single-Linkage (SLINK), and Self-Organizing-Maps (SOM). Various experiments and benchmarking on seven IoT rating datasets from different fields are conducted to assess the performance of the proposed recommender system. The experimental results using both error and prediction metrics indicate that the proposed algorithm outperforms the traditional collaborative filtering approach. Besides, adopting the self-organizing strategy obtains recommendations of significant accuracy as compared to the partitional learning approaches.

Merging visual features and temporal dynamics in sequential recommendation

With the development of social networking and mobile computing technologies, data analysis in the fashion field has increasingly focused on visual features. The main features currently used in the recommendation methods include non-visual user attributes, item attributes, explicit ratings, and implicit feedbacks. How to understand visual features and integrate them with non-visual features becomes the key to building a good recommender system. In this paper, we consider both non-visual text data and visual image data and their time dynamics to build a large-scale recommender system. An advanced visual Bayesian personalized ranking (aVBPR) model is proposed, which integrates three models. Factorized personalized Markov chains (FPMC) model is used to simulate users’ sequence behaviors, intelligent field-aware factorization machine (iFFM) model also put forward by us is used to predict users’ preferences based on non-visual features, and visual Bayesian personalized ranking (VBPR) model is used to analyze users’ visual preferences. We design a learning algorithm based on AdaGrad method to optimize model aVBPR. The high complexity of the model does not affect the performance of the system by adopting multi-thread technology in the implementation of the learning algorithm. Experimental results of two real-world datasets Women's and Men's Clothing & Accessories from Amazon demonstrate that our model can obtain better recommendation results than the recent popular models for Amazon datasets. Although the model is complicated, multi-thread technique can be used to greatly improve the speed of the implementation.

Large Scale Personalized Categorization of Financial Transactions

A major part of financial accounting involves tracking and organizing business transactions over and over each month and hence automation of this task is of significant value to the users of accounting software. In this paper we present a large-scale recommendation system that successfully recommends company specific categories for several million small businesses in US, UK, Australia, Canada, India and France and handles billions of financial transactions each year. Our system uses machine learning to combine fragments of information from millions of users in a manner that allows us to accurately recommend user-specific Chart of Accounts categories. Accounts are handled even if named using abbreviations or in a foreign language. Transactions are handled even if a given user has never categorized a transaction like that before. The development of such a system and testing it at scale over billions of transactions is a first in the financial industry.

Large-scale e-learning recommender system based on Spark and Hadoop

The present work is a part of the ESTenLigne project which is the result of several years of experience for developing e-learning in Sidi Mohamed Ben Abdellah University through the implementation of open, online and adaptive learning environment. However, this platform faces many challenges, such as the increasing amount of data, the diversity of pedagogical resources and a large number of learners that makes harder to find what the learners are really looking for. Furthermore, most of the students in this platform are new graduates who have just come to integrate higher education and who need a system to help them to take the relevant courses that take into account the requirements and needs of each learner. In this article, we develop a distributed courses recommender system for the e-learning platform. It aims to discover relationships between student’s activities using association rules method in order to help the student to choose the most appropriate learning materials. We also focus on the analysis of past historical data of the courses enrollments or log data. The article discusses particularly the frequent itemsets concept to determine the interesting rules in the transaction database. Then, we use the extracted rules to find the catalog of more suitable courses according to the learner’s behaviors and preferences. Next, we deploy our recommender system using big data technologies and techniques. Especially, we implement parallel FP-growth algorithm provided by Spark Framework and Hadoop ecosystem. The experimental results show the effectiveness and scalability of the proposed system. Finally, we evaluate the performance of Spark MLlib library compared to traditional machine learning tools including Weka and R.

Neural Binary Representation Learning for Large-Scale Collaborative Filtering

Integrating hashing into collaborative filtering (CF) has been a promising solution to address the efficiency problem of large-scale recommender systems, i.e., hashing users and items into binary codes and then making recommendations in Hamming space. However, most of the existing hashing methods for CF solely focus on modeling the user-item similarity (a.k.a. preference) but omit the user-user and item-item similarities, which cannot well preserve the original geometry in the vector space. Another tough issue is how to effectively tackle the encoding loss from the continuous vector space to the discrete Hamming space. In this paper, we propose a neural binary representation (NBR) learning approach by combining hashing with a neural network for the large-scale CF tasks. Our NBR problem is formulated as a Hamming similarity loss plus two anchor smoothing terms, which jointly preserve the intrinsic user-item, user-user, and item-item similarities. To compensate the encoding loss introduced by hashing, the anchors for users and items are pre-learned using both user-item interactions and the side-information of users and items through an AutoEncoder model. In particular, to solve the NBR problem, we develop a computationally efficient algorithm, which learns the binary codes in a fast bit-by-bit way and achieves less quantization deviation than the conventional CF hashing schemes. The extensive experiments on three benchmarks validate the superiority of our NBR approach in comparison to the state-of-the-art CF hashing methods.