Content still matters. A machine learning model for predicting news longevity from textual and context features

Abstract

There is an ongoing debate about what is more important in the modern online media newsroom, whether it is the news content and worthiness, or the audience clicks. Using a dataset of over one million articles from five countries (Belarus, Kazakhstan, Poland, Russia, and Ukraine) and a novel machine learning methodology, I demonstrate that the content of news articles has a significant impact on their lifespan. My findings show that articles with positive sentiment tend to be displayed longer, and that high fear emotion scores can extend the lifespan of news articles in autocratic regimes, and the impact is substantial in magnitude. This paper proposes four new methods for improving information management methodology: a flexible version of Latent Dirichlet Allocation (LDA), a technique for performing relative sentiment analysis, a method for determining semantic similarity between a news article and a newspaper's dominant narrative, and a novel approach to unsupervised model validation based on inter-feature consistency.