Saliency based video quality prediction using multi-way data analysis

Abstract

Saliency information allows us to determine which parts of an image or video frame attracts the focus of the observer and thus where distortions will be more obvious. Using this knowledge and saliency thresholds, we therefore combine the saliency information generated by a computational model and the features extracted from the H.264/AVC bitstream, and use the resulting saliency-weighted features in the design of a video quality metric with multi-way data analysis. We used two different multi-way methods, the two dimensional principal component regression (2D-PCR) and multi-way partial least squares regression (PLSR) in the design of a no-reference video quality metric, where the different saliency levels are considered as an additional direction. Our results show that the consideration of the the saliency information leads to more stable models with less parameters in the model and thus the prediction performance increases compared to metrics without saliency information for the same number of parameters.