No-Reference Nonuniform Distorted Video Quality Assessment Based on Deep Multiple Instance Learning

Abstract

Each part of a nonuniform distorted video (NUDV) has a unique distortion degree. When NUDV blocks are used as inputs, traditional machine-learning-based video quality assessment (VQA) methods frequently do not work effectively. Because these methods directly assign the label of the entire video to blocks, causing the unreliability of labels. We creatively propose video bag, a collection of video blocks, to deal with this unreliability. We develop a novel multiple instance learning (MIL) based model, VQA-MIL, which dynamically adjusts the weights by a block-wise attention module and enriches the features of video bags by a MI Pooling layer. Furthermore, we apply the mixup data-augmentation strategy to address the lack of human labels in common video datasets. We test our method on LIVE and CSIQ, and on a relatively large-scale dataset, named NUDV-KT, that we have collected. Results show that our method outperforms popular state-of-the-art no-reference VQA methods on NUDVs.