Evaluating the multivariate visual quality performance of image-processing components

Abstract

The estimation of image quality is a demanding task, especially when estimating different high-quality imaging products or their components. The challenge is the multivariate nature of image quality as well as the need to use naïve observers as test subjects, since they are the actual end-users of the products. Here, we use a subjective approach suitable for estimating the quality performance of different imaging device components with naïve observers—the interpretation-based quality (IBQ) approach. From two studies with 61 naïve observers, 17 natural image contents, and 13 different camera image signal processor pipelines, we determined the subjectively crucial image quality attributes and dimensions and the description of each pipeline's perceived image quality performance. We found that the subjectively most important image quality dimensions were color shift/naturalness, darkness, and sharpness. The first dimension, which was related to naturalness and colors, distinguished the good-quality pipelines from the middle- and low-quality groups, and the dimensions of darkness and sharpness described why the quality failed in the low-quality pipelines. The study suggests that the high-level concept naturalness is a requirement for high-quality images, whereas quality can fail for other reasons in low-quality images, and this failure can be described by low-level concepts, such as darkness and sharpness.