Images Inpainting Quality Evaluation Using Structural Features and Visual Saliency

Abstract

Despite the extensive research on developing robust image inpainting algorithms in recent years, there are almost no objective metrics for the quality assessment of inpainted images currently. Inspired by the feature coherence in the inpainted image and the human visual perception mechanism, this paper proposes an image inpainting quality assessment (IIQA) that takes into account both visual saliency and structural features. First, the quality issues associated with image inpainting are categorized into three aspects: incoherent structure, unreasonable texture, and other results that are inconsistent with human visual perception. These quality problems are further expressed as “regions of interest” and extracted by the visual saliency method using the natural statistics model. Subsequently, the structural features are computed based on the nonlinear diffusion of the horizontal and vertical gradient field of the inpainted image. Finally, the IIQA metric incorporates brightness, gradient similarity, structural similarity, and visual saliency is established. The quality evaluation process is conducted by comparing each patch within the inpainted region with its best match from the known region. The quantitative experimental results demonstrate the effectiveness of the proposed method, especially for images with structural discontinuity. A comparative study also shows that the Spearman rank order correlation coefficient of our method achieves 0.875 on certain databases, which outperforms existing IIQA metrics.