Abstract
For many practical applications, it is essential to address both geometric corrections and missing information reconstruction of face images and natural images. However, it is unfavorable to separate the problem into two sub-tasks due to error accumulations of sequential tasks. In this paper, we propose a novel robust missing information reconstruction framework via deep feature transformations to simultaneously address both geometric corrections and image completion. Specifically, our proposed framework realizes multiple channel spatial transformations to tackle geometric corrections, and address image completion through non-linear features projections. The flow of our framework includes deep feature extraction, feature enhancement, feature projection, and feature refinement, where deep features are extracted and learnt to achieve robust image completion. Experimental results show the superior performance of our framework for both face images and natural images in various databases. Compared with the conventional approaches approach to split the problem into two sub-tasks, including image inpainting and spatial transformation, our proposed framework achieves a number of advantages, including i) an unified framework to automatically correct the geometric distortions and to reconstruct the missing information simultaneously and ii) achieving much better visual quality for those recovered images.
Highlights
Over the past decades, significant progress has been achieved to develop new deep learning networks that achieve improved solutions for a range of computer vision problems, including image classification [1]–[3], face recognition [1], [4], [5], object detection [6]–[8], semantic segmentation [9], single image super-resolution (SISR) [10]– [16], and image inpainting [17]–[30]
To enable the proposed deep feature transformation framework to work in terms of deep features, we firstly apply a VGG19 network to extract deep features from the input images, and design a feature refinement unit (FRU) with a total of 18 layers to ensure that the multi-channel feature maps generated can have sufficient details and hierarchies for optimized estimation of transformation parameters in feature domain
We propose adding a Feature Refinement Unit (FRU) into the localization network to further process the deep features and extract fine details in hierarchies to optimize the estimation of feature transformation parameters
Summary
Significant progress has been achieved to develop new deep learning networks that achieve improved solutions for a range of computer vision problems, including image classification [1]–[3], face recognition [1], [4], [5], object detection [6]–[8], semantic segmentation [9], single image super-resolution (SISR) [10]– [16], and image inpainting [17]–[30]. As STN estimates the transformation parameters and resample the input image using bilinear interpolation in pixel domain, its output often becomes blurred compared with its original input image [37] To this end, we propose a robust missing information reconstruction framework to simultaneously address the geometric corrections and estimation of masked regions of image inpainting. In comparison with the existing image inpainting methods, our proposed missing information reconstruction framework achieves the advantage that both the training cost and the learning complexity is much lower via less number of model parameters;.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
