Predicting video popularity based on video covers and titles using a multimodal large-scale model and pipeline parallelism

Abstract

In the era of traffic, controlling traffic is equivalent to mastering influence and economic benefits. At the video level, under the premise of the same video content, it is very important to study what kind of cover and title can be more attractive to people. Unlike most previous studies that focused on YouTube videos, our data came from Bilibilis videos. This paper tried to use two neural network models, ViT and Bert, combined with GPipe and backend fusion multimodal data fusion methods, to predict the possible click-through rate and popularity of a specific video based on its existing video cover and title. In the process, we switched to different visual and language models to complete the same training task, with the goal of comparing the impact of different models on the results. By adjusting the weight of two models, we finally achieved a good result of up to 62% accuracy.