Image inpainting based on sparse representations with a perceptual metric

Takahiro Ogawa,Miki Haseyama

doi:10.1186/1687-6180-2013-179

Takahiro Ogawa, Miki Haseyama

Open Access

https://doi.org/10.1186/1687-6180-2013-179

Copy DOI

Abstract

This paper presents an image inpainting method based on sparse representations optimized with respect to a perceptual metric. In the proposed method, the structural similarity (SSIM) index is utilized as a criterion to optimize the representation performance of image data. Specifically, the proposed method enables the formulation of two important procedures in the sparse representation problem, 'estimation of sparse representation coefficients’ and 'update of the dictionary’, based on the SSIM index. Then, using the generated dictionary, approximation of target patches including missing areas via the SSIM-based sparse representation becomes feasible. Consequently, image inpainting for which procedures are totally derived from the SSIM index is realized. Experimental results show that the proposed method enables successful inpainting of missing areas.

Highlights

In the field of image processing, there exist many studies on image restoration/enhancement such as image denoising [1,2,3], image deblurring [4,5], and image inpainting [6]
We present an inpainting method based on sparse representations optimized with respect to a perceptual metric
We mainly explain the details of the experiments and the comparative methods

Summary

Introduction

In the field of image processing, there exist many studies on image restoration/enhancement such as image denoising [1,2,3], image deblurring [4,5], and image inpainting [6]. By deriving the sparse representation of target patches including missing areas based on the generated dictionary, inpainting based on the SSIM index is realized. The representation coefficients and the atoms of the dictionary matrix are calculated in such a way that the SSIM-based approximation performance becomes the highest. This means that the cost function ||Y − DX||2F in Equation 2 is replaced with that of the SSIM index. The proposed method performs the sparse representation of the target patch to estimate the missing intensities. From the approximation results obtained by the above sparse representation, the proposed method outputs the estimated intensities within the missing areas of the target patch. By iterating the patch selection based on the patch priority and its SSIM-based missing area reconstruction, we can inpaint the whole missing areas within the target image

Image inpainting via SSIM-based sparse representation

Calculation of the optimal vector xi

Conclusions