BCE4ZSR: Bi-encoder empowered by teacher cross-encoder for zero-shot cold-start news recommendation

Abstract

In the realm of news recommendations, the persistent challenge of the cold-start problem continues to impede progress. Existing approaches rely heavily on information exchange between news articles and users to personalize news recommendations and have struggled to adapt to users and news articles without historical interaction data. In this paper, we introduce BCE4ZSR, a novel framework that leverages a rarely utilized zero-shot approach to effectively tackle the cold-start problem in news recommendations. The proposed approach consists of two main steps: First, we generate embeddings for inference with a sentence transformer (a bi-encoder). In the second step, a fine-tuned transformer model is augmented during the training phase to distil the bi-encoder with knowledge from the cross-encoder model using a student–teacher framework. As a result of this synergy, the cross-encoder serves as a teacher, imparting its knowledge to the bi-encoder. The proposed technique can be applied to any neural news recommender system and is empirically evaluated in both cold-start and regular user-news interaction situations. Experiments on real-world benchmark datasets (MIND-small and MIND-large) indicated that BCE4ZSR outperformed the baseline methods in terms of nDCG@k, AUC, and MRR. Specifically, AUC improvement of 1.5%–6% & 2.2%–7.93%; MRR improved by 1%–13.85% & 0.9%–14.72%; nDCG@5 improved by 4.9%–18.79% & 2.1%–16.44%; and nDCG@10 improved by 1.7%–15.18% & 1.9%–14.24% in the cold start and warm user scenarios respectively, proving the superiority of our model compared to baseline methods.