Video quality assessment using motion-compensated temporal filtering and manifold feature similarity.

Yang Song,Mei Yu,Feng Shao,Gangyi Jiang,Zongju Peng,You Yang

doi:10.1371/journal.pone.0175798

Abstract

Well-performed Video quality assessment (VQA) method should be consistent with human visual systems for better prediction accuracy. In this paper, we propose a VQA method using motion-compensated temporal filtering (MCTF) and manifold feature similarity. To be more specific, a group of frames (GoF) is first decomposed into a temporal high-pass component (HPC) and a temporal low-pass component (LPC) by MCTF. Following this, manifold feature learning (MFL) and phase congruency (PC) are used to predict the quality of temporal LPC and temporal HPC respectively. The quality measures of the LPC and the HPC are then combined as GoF quality. A temporal pooling strategy is subsequently used to integrate GoF qualities into an overall video quality. The proposed VQA method appropriately processes temporal information in video by MCTF and temporal pooling strategy, and simulate human visual perception by MFL. Experiments on publicly available video quality database showed that in comparison with several state-of-the-art VQA methods, the proposed VQA method achieves better consistency with subjective video quality and can predict video quality more accurately.

Highlights

The rapidly growing popularity of such digital consumer electronic devices as smartphones and portable computers has rendered video applications ubiquitous in our daily lives
group of frames (GoF) are first decomposed into a temporal high-pass component (HPC) and a temporal low-pass component (LPC) by motion-compensated temporal filtering (MCTF)
According to the similarity measurement defined in SSIM (Structural Similarity), the manifold feature similarity for each block in temporal LPC can be acquired as Eq (9)

Summary

Introduction

The rapidly growing popularity of such digital consumer electronic devices as smartphones and portable computers has rendered video applications ubiquitous in our daily lives. The image quality of each frame clearly contributes considerably to overall video quality Such methods overlook the importance of temporal information, which limits their effectiveness. Seshadrinathan et al [7] considered motion information as a video feature and proposed motion-tuned, spatio-temporal quality assessment of natural video (MOVIE). Zhang et al [9] exploited the visual masking effect to process the human perception of distortion in videos, and proposed a perception-based VQA method. An asymmetric temporal pooling strategy is adopted to obtain an overall video quality This newly proposed VQA method takes the following unique features: 1. To ensure the VQA method incorporate with human visual characteristics, we use manifold learning as a perceptual approach to extract features.

Materials and methods

Â MFref ðiÞ Â MFdisðiÞ þ C1 MFref 2ðiÞ þ MFdis2ðiÞ þ C1

Â PCrHefPC Â PCdHisPC þ C2 ðPCrHefPCÞ2 þ ðPCdHisPCÞ2 þ C2

Results and discussion

Conclusions