On the trade-off between ranking effectiveness and fairness

Abstract

This paper addresses the problem of maximizing the effectiveness of the ranking produced by information retrieval or recommender systems and at the same time maximizing two fairnesses, that of the group and that of the individual. The context of this paper is therefore that of access to information carried out by users, who aim to satisfy their own information needs, to documents produced by authors and curators who aim to be exposed in a fair manner, i.e. without discriminating between groups nor individuals. The paper describes a general method based on the spectral decomposition of mixtures of symmetric matrices, each of which represents a variable to be maximized, and experiments conducted with the use of a test collection. The method described in this paper has explained if and how the trade-offs between effectiveness, group fairness and individual fairness manifest themselves. The experimental results show that maintaining an acceptable level of effectiveness and fairness at the same time is feasible and (b) the trade-offs exist but the order of magnitude of the variations depends on the measure of effectiveness used and therefore by what the user’s model of access to information is as well as on the fairness measures and therefore on how authors or editors should be exposed.