A hybrid network of CNN and transformer for subpixel shifting-based multi-image super-resolution

Abstract

Multi-image super-resolution (MISR) involves synthesizing multiple low-resolution (LR) images captured from various perspectives or times to reconstruct a single high-resolution (HR) image. Consequently, the integration of multiple LR views into a HR image with detailed features becomes a challenging problem. This study introduces a super-resolution reconstruction method for subpixel-shifted (SPS) images, employing a synergistic strategy that combines both hardware and software approaches. Our objective is to notably enhance image resolution and quality by effectively addressing limitations in existing hardware and software methods for image resolution enhancement, thereby achieving more accurate and high-quality image reconstruction. In the first stage, a series of SPS images with fixed shifts are acquired through hardware, enabling high-precision subpixel shift technology. In the second stage, we introduce an SP-MISR network model, leveraging a hybrid of convolutional neural networks and transformer architecture, to utilize the subpixel shift information for generating HR images from SPS images. Experimental results demonstrate the superiority of the proposed method over existing technologies across multiple standard datasets, effectively improving both image resolution and reconstruction quality. This study provides novel perspectives and methodologies for the field of image processing, demonstrating the potential of integrating hardware precision with advanced AI-based image processing techniques to enhance super-resolution imaging.