Combined frequency and spatial domain-based patch propagation for image completion

Abstract

Film and photography archives now have an accelerated rate of degradation. Because the preservation of cultural heritage plays an important role in our society, photograph and film restoration has recently drawn a substantial amount of attention. In this paper, an approach that involves exemplar-based inpainting, aimed at determining patch priority and patch matching, is proposed. Different image regions have different levels of importance for vision perception; hence, a priority score must be assigned. Patch priority is calculated by the energy of its distributed cosine transform coefficients (DCT term) and the edge term. The edge term prioritizes the edge patches and the energy of the DCT coefficients of the patch is used as a discriminator for patches with similar edge terms. Patch inpainting is performed by assessing the similarity between the patches in such a manner that the similarity measure is consistent with human visual judgment. Therefore, a structure-based similarity measure is developed. Further, the interpolated missing pixels at the patch are also considered for applying the structure-based patch matching criteria in finding the candidate patch. Experimental results on damaged digitized photographs and natural images are presented, which demonstrate the effectiveness of the image completion framework for tasks such as scratch/text, object removal and image inpainting.