The impact of first recommendations based on exploration or exploitation approaches in recommender systems' learning

Abstract

The current advances have recognized the online recommendation task as a Multi-Armed Bandit (MAB) problem and proposed several algorithms by combining it with Collaborative Filtering (CF). In general, their effectiveness is related to their practical ability to handle the trade-off between selecting the most relevant item (exploitation) or choose an unexpected item to add more knowledge to the system (exploration). However, we observed that a naive failure has been introduced in the main CF bandit models, which have intrinsically reduced itself to pure exploitation of the information known a prior by the system (e.g., items' popularity) or a pure exploration of the items in the first user's interaction. Conversely, we hypothesize this trade-off should be considered since the first user's interactions to improve the system's effectiveness. Thus, we propose a new methodology to measure the impact of this problem. This methodology has two main phases. In phase 1, traditional non-personalized methods resemble pure-exploitation and pure-exploration to feed the learning of all MAB models. Then, analyzing the MAB effectiveness in phase 2, we may determine the best approach to be considered in the first user's interactions. Moreover, we also propose and add a new strategy that combines exploration and exploitation in phase 1. Indeed, our experimental results show that the combination of exp-exp at first user's interactions improves the system's effectiveness without adding more effort to users.