Churning while Learning: Maximizing User Engagement in a Recommendation System

Abstract

Online media platforms such as Spotify, YouTube Music, and NetEase Cloud Music rely on long-term user engagement for revenue generation. The primary operational level under their control is content recommendation (i.e., what content to recommend to various users), where the right recommendation can induce users to further interact with the platform, sign-up for premium memberships, view ads, and more. However, given constantly updating supply (i.e., new content being created) and heterogeneous user behavior, optimal recommendation is challenging. It requires a careful balance between exploration (understanding the efficacy of new content) and exploitation (recommending well understood existing content). Motivated by a real-world impressions level dataset from NetEase Cloud Music, we propose a two-period model for the platform recommendation problem, with the aim of maximizing long term user engagement. Our model captures two key features of this marketplace: (1) supply-side learning (i.e., platform learning the efficacy of new content) and (2) demand-side heterogeneous churning (i.e., different users churning at different rates as a function of both their engagement and their types). We use our two-period model to understand the interplay between churning while learning and how it impacts the long-term user engagement. In addition to characterizing the structure of the optimal learning policy for the platform recommendation problem (i.e., which users to experiment on), we numerically illustrate our key findings on a wide range of parameters. We find that accounting for heterogeneous churning behavior while learning the efficacy of new content can lead to improvements of up to 14% in the long-term user engagement.