SISRSet: Single image super-resolution subjective evaluation test and objective quality assessment

Abstract

With the outstanding performance of deep learning based single image super-resolution (SISR) methods, some widely used quality evaluation metrics such as peak signal to noise ratio (PSNR) and structure similarity index (SSIM) have become more and more difficult to meet the assessment requirements of SISR methods, especially in terms of their consistency with subjective visual perception. To deal with this super-resolution (SR) image quality assessment (IQA) issue, it calls for a specialized objective evaluation metric based on the visual characteristics of SR images and the human visual system (HVS). Notwithstanding, there is a lack of practical databases for analysis. In this paper, the SISR subjective evaluation tests are conducted to build an SISRSet database, which requires a large number of reconstructed images generated by various SR algorithms. The statistical analyses of SISRSet reveal that high-frequency texture details play an important role in the performance evaluation of SR algorithms. Inspired by the origination selectivity mechanism (OSM) and the internal generative mechanism (IGM) of the HVS, a visual content prediction model (VCPM) is proposed to measure different visible structural contents of SR images, especially the texture details. Finally, a novel SISR quality assessment metric is devised based on the VCPM similarity comparison between the references and SR images. The experimental results demonstrate that the proposed SISR-IQA metric does well in the performance evaluation of SISR methods and well correlates with those by the subjective evaluation.