Revised Spatial Transformer Network towards Improved Image Super-resolutions

Abstract

In this paper, we propose two approaches to generate a high resolution (HR) image from a low resolution (LR) version. It is commonly referred as single-image super-resolution(SISR). Our approaches are inspired by the spatial transformer (ST) module and Very Deep Convolutional Network (VDSR). The spatial transformer module is the neural network originally used for geometric transformations of images, while the VDSR is for image super-resolution. In the first approach, we propose to add the ST module with the VDSR to generate HR images. The use of the spatial transform with VDSR makes the network more robust to the geometric transformations. We propose the second approach to replace the convolutional neural network (CNN) used in the ST module with VDSR network. The replacement of CNN leads to improvements in the performance of ST module. The simulation results confirm that the feasibility of combing ST module and VDSR for super-resolution reconstruction, where the performance of the combination of ST module and VDSR is comparable to the VDSR alone in terms of Peak signal-to-noise ratio (PSNR) and structural Similarity Index Measurement (SSIM). Hence, the revised spatial transformer network can be used in the future for simultaneous geometric transformation and image super-resolution, which solve the practical applications of image super-resolution in real life.