The Role of Structure and Textural Information in Image Utility and Quality Assessment Tasks

Abstract

Abstract The perceptual process of images is hierarchical. Human tends to first perceive global structural information such as shapes of objects and further focus on local regional details such as texture. Furthermore, it is widely believed that structure information plays the most important role in task of utility assessment and quality assessment, especially in new scenarios like free-viewpoint television, where the synthesized views contain geometric distortion around objects. We thus hypothesize that the degradation of structural information in an image is more annoying for human observers than the one of the textures in certain application scenarios. In order to confirm our hypothesis, a bilateral filtering based model (BF-M) is proposed referring to a recent subjective perceptual test. In the proposed model, bilateral filters are first utilized to separate structure from the texture information in images. Afterward, features that capture object properties and features that reflect texture information were extracted from the response and the residual of bilateral filtering separately. A contour, a shape related and a texture based estimator are then proposed with the corresponding extracted features. Finally, the model is designed by leveraging the three estimators according to target tasks. With the task-based model, one can then investigate the role of structure/texture information in certain task by checking the correspondence optimized weights assigned to the estimators. In this paper, the hypothesis and the performance of the BF-M is verified on CU-Nantes database as utility estimator and on SynTEX, IRCCyN/IVC-DIBR databases as quality estimator. Experimental results show that (1) structure information does play greater role in several tasks; (2) the performance of the BF-M is comparable to the state-of-the art utility metrics as well as the quality metrics designed for texture synthesis and views synthesis. It is thus validated that the proposed model can also be applied as a task-based parametric image metric.