Which is better? A modularized evaluation for topic popularity prediction

Abstract

The purpose of topic popularity prediction is to predict whether a topic on the Internet will become popular. Various elegant models have been proposed for this problem. However, different datasets and evaluation metrics they use lead to low comparability. In this paper, we conduct a comprehensive survey, propose a modularized evaluation scheme for evaluating the models and apply it to existing methods. Our scheme has four modules: categorization; qualitative evaluation on several metrics; quantitative experiment on real world data; and final ranking with risk matrix and MinDis to reflect performances under different scenarios. Furthermore, we analyze the efficiency and contribution of features used in feature-oriented methods. Our work helps users compare models and select appropriate ones for different requirements.