Progressive feature generation with mask awareness for image inpainting

Abstract

Most existing image inpainting methods have achieved remarkable progress in repairing small image defects. However, when the holes are large, their filling contents suffer from structural distortion and center blur due to the weak correlation between known and unknown pixels. In this paper, we propose a Progressive Feature Generation (PFG) network which is mask awareable during the process of filling irregular holes. Specifically, in order to strengthen the constraint for the hole center, we propose dynamic partial convolution which can adaptively adjust the inpainting proportion according to mask ratio in each recurrence. To synthesize high-quality features in the feature generation phase, two parallel encoders are designed which could effectively improve the capability of model to restore large holes. We argue that during feature merging, the signals generated earlier at the same location are more credible. To this end, we develop the sub-regional weighted merging(SWM) method for PFG-Net to accurately merge the feature maps generated during the whole reconstruction process. Experiments on CelebA, Places2, and Paris StreetView datasets show that our model has excellent performance especially for large holes.