Revisiting the robustness of spatio-temporal modeling in video quality assessment

Abstract

Video quality assessment (VQA) is a non-trivial task since accurately predicting video quality is of great importance in real applications. Most VQA models focus on designing efficient spatio-temporal modeling modules for improving their performance on the closed-sets while ignoring the core problem, i.e., the robustness of spatio-temporal modeling on predicting video quality. In this paper, we present an empirical study on spatio-temporal modeling in VQA by disturbing temporal information. To this end, we elaborately construct two video quality databases named Motion-Free and Motion-Interrupted, respectively. Specifically, the videos in the former database are without implicit or explicit temporal information, while the temporal information of the videos in the latter database is perturbed artificially. Then, we conduct a comprehensive study on the constructed databases by testing the state-of-the-art VQA models, and explore the robustness of spatio-temporal modeling in VQA. We find that the spatio-temporal modeling modules are prone to overlook motion interruption, in other words, the VQA models cannot handle the global flickering problem in videos. Moreover, they become ineffective when the temporal information is completely lost.