News recommendation system with category awareness based on pre-trained Electra

Abstract

A news recommendation system is crucial to help users locate interesting news and relieve information overload. How to accurately model news text is the key to a news recommendation system. Traditional news recommendation systems use news encoders that directly model the text of all historical reading news. However, pre-trained language model (PLM) trained on large-scale corpora often have more robust reading comprehension capabilities and can extract more profound implicit expressions of news texts. Users have their preferences when reading different categories of news. Using a large number of training samples to learn such preferences will consume many computing resources. Adding category awareness to the news recommendation system can efficiently and accurately model users and build a personalized News recommendation system. Therefore, this paper uses Electra as a news encoder to capture a more robust contextual implicit representation of news texts. At the same time, additional weights are used to model different news categories. On this basis, users' behavior logs are added to build a news recommendation model. The cosine similarity is used as a loss function to obtain the recommendation probability of candidate news. Experiments show that the AUC of this method is 2.37% higher than the current state-of-the-art methods on the Mind-small datasets.