Is diversity optimization always suitable? Toward a better understanding of diversity within recommendation approaches

Abstract

The diversity of the item list suggested by recommender systems has been proven to impact user satisfaction significantly. Most of the existing diversity optimization approaches re-rank the list of candidate items during a post-processing step. However, the diversity level of the candidate list strongly depends on the recommender system used. Hence, applying the same post-processing diversification strategy may not be as effective for different recommendation approaches. Moreover, individual users’ diversity needs are usually ignored in the diversification post-processing.This article aims at providing an in-depth analysis of the diversity performances of different recommender systems. To the best of our knowledge, it is the first study to systematically compare diversity performances of the main types of recommendation models using benchmark datasets in different domains (movie, anime and book). Semantics related to items may be considered a key factor in measuring diversity within recommender systems. In this study, we leverage support from the knowledge engineering domain and take advantage of resources such as Linked Data and knowledge graphs, to assert the diversity of recommendations. We also propose a variant of the classic diversification post-processing objective that allows to take into account specific users’ diversity needs. We measure the adequacy between the diversity levels a recommender system suggests to its users and those of users’ profiles with the R2 coefficient of determination.Our study indicates that: (1) none of the tested recommender systems, even the most recent ones, provides items with levels of diversity that suit user profiles (R2<0.2); (2) the classic post-processing diversification approach may lead to over-diversification compared to users’ diversity needs and (3) the diversity adjustment that accounts for user profiles has more benefits (greater R2 and smaller accuracy loss). All the source code and datasets used in our study are available to ensure the reproductibility of the study.