The crowd against the few: Measuring the impact of expert recommendations

Abstract

A large amount of research on recommender systems has focused on improving the accuracy of suggestions in offline settings. However, this focus and the commonly used techniques can lead to a “filter bubble”, severely limiting the diversity of content discovered by users. Several offline studies show that this can be mitigated by using experts for recommendation. In contrast to standard recommender systems, experts are able to generate more diverse recommendations and increase the novelty of given suggestions. They can be used in missing-data or cold-start scenarios and reduce noise in the users' ratings. This paper examines the impact of employed experts' recommendations on user behavior for a real-world recommender system on a popular video-on-demand website, provided by a large television network. We study whether the potential benefits of experts lead to differences in user behavior, user perceptions and properties of given recommendations (e.g., diversity). We find that enriching a state-of-the-art system with the suggestions of employed experts can significantly increase platform use. Even though expert recommendations are used less frequently and are less successful than expected, users watch a greater number of clips, use more recommendations, and come back to the website more frequently when they receive expert suggestions. When searching for other influencing factors, we find that experts generate more diverse recommendations and improve the taste coverage of the system keeping user satisfaction unaffected. In summary, our results show large benefits of using employed experts and have implications for the design and use of recommender systems in real-world scenarios.