Exploiting personalized calibration and metrics for fairness recommendation

Abstract

Recommendation systems are used to suggest items that users can be interested in. These systems are based on the user preference historic to create a recommendation list with items that have the higher similarity with the user’s interests, in order to achieve the best possible user’s satisfaction, which is usually measured as recommendation precision. However, the search for the best precision can cause some side effects such as overspecialization, few diversity and miscalibration of genres, classes and niches. Calibration provides fairer recommendations, which respect the genre proportionality on the user’s preferences, avoiding overspecialization. This article aims to explore ways to balance the trade-off weight between precision and calibration based on divergence measures, as well as to propose metrics to evaluate the calibration in the suggested list. The proposed system works in a post-processing step and does not depend on a specific recommender algorithm or workflow. For this purpose, we evaluate six recommender algorithms applied in the movie domain, analyzing variations of three fairness measures, two personalized trade-off weights and eleven constant weights. To understand the results we use the precision, the reciprocal rank and two proposed metrics. The results indicate that the trade-off formulation of personalized weights obtains better results when used to compare the recommendation lists using matrix factorization-based approaches on Movielens dataset. In addition, the calibration also impacts the precision and fairness of all considered algorithms used in evaluation.