Metric Optimization and Mainstream Bias Mitigation in Recommender Systems

Abstract

Recommender Systems have drawn extensive attention in recent decades. The most important factor to make a recommender succeed is user satisfaction, which is largely reflected by the recommendation accuracy. In accordance, this thesis dives into recommendation accuracy from two different perspectives that, unfortunately, are at tension with each other: achieving the maximum overall recommendation accuracy, and balancing that accuracy among all users. The first part of this thesis focuses on maximizing the overall recommendation accuracy Li et al. [2021a]. This accuracy is usually evaluated with some user-oriented metrics tailored to the recommendation scenario. However, recommendation models could be trained to maximize some other generic criteria that do not necessarily align with the criteria ultimately captured by the metric above. Recent research usually assumes that the metric used for evaluation is also the metric used for training. We challenge this assumption, mainly because some metrics are more informative than others. Indeed, we show that models trained via the optimization of a loss inspired by Rank-Biased Precision (RBP) tend to yield higher accuracy, even when accuracy is measured with metrics other than RBP. However, the superiority of this RBP-inspired loss stems from further benefiting users who are already well-served, rather than helping those who are not. This observation inspires the second part of this thesis, where our focus turns to helping nonmainstream users, who are difficult to recommend to either because of the lack of data for effective modeling or a niche taste that is hard to match similar users. Our first effort consists in using side data, beyond the user-item interaction matrix, so that users and items are better represented. This will be of benefit especially for the non-mainstream users, for which the user-item matrix alone is ineffective. We propose Neural AutoEncoder Collaborative Filtering (NAECF) Li et al. [2021b], an adversarial learning architecture that, in addition to maximizing the recommendation accuracy, leverages side data to preserve the intrinsic properties of users and items and show that NAECF leads to better recommendations specially for non-mainstream users, while at the same time there is a marginal loss for the mainstream ones. Our second effort Li et al. [2023] consists in explicitly focusing more on non-mainstream users in a recommendation model. In particular, we propose a mechanism based on cost-sensitive learning that weighs users according to their mainstreamness, so that they get more attention during training. The result is a recommendation model tailored to non-mainstream users, that narrows the accuracy gap, and again at negligible cost to the mainstream users. Awarded by : Delft University of Technology, Delft, The Netherlands on 14 November 2023. Supervised by : Alan Hanjalic and Julián Urbano. Available at : https://doi.org/10.4233/uuid:235a5f5c-5e18-4e67-a8e6-84a7e310db12.