Abstract
Deep learning has brought unprecedented progress to image inpainting. However, the existing methods often generate images with blurry textures and distorted structures because they may either fail to maintain semantic consistency or restore fine-grained textures. In this paper, we propose a two-stage adversarial model to further improve the accuracy of the structure and details of image inpainting. Our model splits the inpainting task into two parts: semantic structure reconstructor and texture generator. In the first stage, we first utilize the semantic structure map based on the unsupervised segmentation to train the semantic structure reconstructor, which completes the missing structures of the inputs and maintains consistency between the missing part and the overall image. In the second stage, we introduce the spatial-channel attention (SCA) module to obtain the fine-grained textures. The SCA module strengthens the capability to obtain information from the long-distance pixel and different channels of the model. Furthermore, we propose a spatial-channel loss to stabilize the network training process and improve visual effects. Finally, we evaluate our model over the publicly available datasets CelebA, Places2, and Paris StreetView. When the inpainting tasks involved in large-area defects or heavy structure, the experimental results show that our method has a higher inpainting quality than the existing state-of-the-art approaches.
Highlights
Image inpainting, aim to restoring missing regions according to the rest of the image in image processing, has been widely used in image editing, such as removing unwanted objects and editing contents of images
To solve the problem of difficulty in obtaining information from distant pixels result from the limitation of the receptive fields of the convolution kernel and make full use of the feature information of different channels, we introduce the spatial-channel attention (SCA) modules to make each pixel is calculated by the elementwise sum in the spatial and channel information in texture generator
We first introduce a new method to restore the semantic structure map based on the unsupervised segmentation and the spatial and channel attention module (SCA module) to complete the image repair
Summary
Aim to restoring missing regions according to the rest of the image in image processing, has been widely used in image editing, such as removing unwanted objects and editing contents of images. The artist usually first determines the area of the object and further fills in the details of different areas in the creative process of painting To solve these problems of the over-smoothed boundaries and texture artifacts, we propose a novel method to accurately extract semantic structure information of images. Yu et al propose the reason for the image with distorted structures and blurry textures is the ineffectiveness of convolutional neural networks in explicitly borrowing or copying information from distant spatial locations They introduce the contextual mechanism to enhance the model of long-term correlations. Wang et al [31] propose a network to further extract image features by combining information from different receptive fields In network structure, these papers [11], [32]–[34] have introduced the two-stage network.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
