Review of subjective quality assessment methodologies and standards for compressed images evaluation

Abstract

Lossy image compression algorithms are usually employed to reduce the storage space required by the large number of digital pictures that are acquired and stored daily on digital devices. Despite the gain in storage space, these algorithms might introduce visible distortions on the images. However, users typically value the visual quality of digital media and do not tolerate any distortion. Objective image quality assessment metrics propose to predict the amount of such distortions as perceived by human subjects, but a limited number of studies have been devoted to the objective assessment of the visibility of artifacts on images as seen by human subjects. In other words, most objective quality metrics do not indicate when the artifacts become imperceptible to human observers. An objective image quality metric that assesses the visibility of artifacts could, in fact, drive the compression methods toward a visually lossless approach. In this paper, we present a subjective image quality assessment dataset, designed for the problem of visually lossless quality evaluation for image compression. The distorted images have been labeled, after a subjective experiment held with crowdsourcing, with the probability of the artifact to be visible to human observers. In contrast to other datasets in the state of the art, the proposed dataset contains a big number of images along with multiple distortions, making it suitable as a training set for a learning-based approach to objective quality assessment.