Pull–push: a measure of over- or underpersonalization in recommendation

Abstract

A recommender system imposes differences between users, by presenting to them different recommendation lists, which they respond to, resulting in different “reaction” lists. Comparison of the differences in the recommendation and reaction lists can indicate different user states. Users can approve the imposed difference, end up narrowing the difference between them (pulling each other closer) by consuming more of the items in common or enlarge the difference between them (pushing each other further apart) by consuming the items not in common. When users do not approve the differences, they are either in a push state (implicitly disapproving under-personalization) or in a pull state (implicitly disapproving over-personalization). We offer the pull–push metric to quantify the magnitude of pull or push—measures of disapproval by the users of, respectively, over-personalization and under-personalization. Application on simulated datasets shows that users can push each other away up to disjoint sets or pull each other closer up to identical sets. On real-world datasets, we find that the particular recommender system was under-personalizing its recommendations. We show how the pull–push metric can be merged with another metric of personalization to come up with a measure of the potential for improvement in a recommender system and discuss its relationship to popularity bias.