Unsupervised Super-Resolution Research Articles

A novel single image super-resolution reconstruction model based on edge-enhanced Siamese generative adversarial networks

Abstract Currently, most image super-resolution (SR) reconstruction algorithms are supervised, improving the overall visual quality of images based on a large amount of paired low-resolution (LR) and high-resolution (HR) image. However, collecting a large number of paired datasets is challenging, and the quality of the dataset can influence the reconstruction results. Although unsupervised methods have gained significant performance based on synthetic datasets, their effectiveness is relatively poor when applied to real-world or natural data. Focusing on those aforementioned issues, a novel image SR reconstruction algorithm of Edge-enhanced Siamese Generative Adversarial Network (EeSiGAN) is proposed. EeSiGAN belongs to the unsupervised category and does not require the involvement of HR images. Firstly, employing the Siamese GAN as the overall framework ensures the continuous consistency of image feature maps through the collaborative effect of support and main networks. Secondly, fusing the results of the two branches by using the Multi- scale Channel Attention Feature Fusion (MsCAFF) module. In addition, an Edge- enhanced Feature Distillation Block (EFDB) is introduced to enhance edge information in images and optimize the capability in feature extraction of network. Finally, Directional Variance Attention (DiVA) mechanism is used to obtain long-range spatial dependencies and simultaneously utilize inter-channel dependencies to achieve a more discriminative representation, thereby enhancing the overall quality of the recovered images. Extensive experimental results on synthetic and real datasets demonstrate that compared to other state-of-the-art unsupervised SR networks, the proposed EeSiGAN algorithm exhibits better performance in reconstructing images, producing clearer details and textures in the enlarged images.

Implicit neural representations for unsupervised super-resolution and denoising of 4D flow MRI

Background and Objective: 4D flow magnetic resonance imaging provides time-resolved blood flow velocity measurements, but suffers from limitations in spatio-temporal resolution and noise. In this study, we investigated the use of sinusoidal representation networks (SIRENs) to improve denoising and super-resolution of velocity fields measured by 4D flow MRI in the thoracic aorta.Methods: Efficient training of SIRENs in 4D was achieved by sampling voxel coordinates and enforcing the no-slip condition at the vessel wall. A set of synthetic measurements were generated from computational fluid dynamics simulations, reproducing different noise levels. The influence of SIREN architecture was systematically investigated, and the performance of our method was compared to existing approaches for 4D flow denoising and super-resolution.Results: Compared to existing techniques, a SIREN with 300 neurons per layer and 20 layers achieved lower errors (up to 50% lower vector normalized root mean square error, 42% lower magnitude normalized root mean square error, and 15% lower direction error) in velocity and wall shear stress fields. Applied to real 4D flow velocity measurements in a patient-specific aortic aneurysm, our method produced denoised and super-resolved velocity fields while maintaining accurate macroscopic flow measurements.Conclusions: This study demonstrates the feasibility of using SIRENs for complex blood flow velocity representation from clinical 4D flow, with quick execution and straightforward implementation.

Deep-learning image enhancement and fibre segmentation from time-resolved computed tomography of fibre-reinforced composites

Monitoring the microstructure and damage development of fibre-reinforced composites during loading is crucial to understanding their mechanical properties. Time-resolved X-ray computed tomography enables such an in-situ, non-destructive study. However, the photon flux and fibre-matrix contrast limit its achievable spatial and temporal resolution. In this paper, we push the limits of temporal and spatial resolution for the microstructural analysis of unidirectional continuous carbon fibre-reinforced epoxy composites by establishing a new pipeline based on CycleGAN for unsupervised super-resolution and denoising and U-Net-id for individual fibre segmentation. After illustrating the benefits of a 3D CycleGAN over a 2D one, we show that data enhanced by this pipeline can yield similar segmentation quality to that of a slow-acquisition, high-quality scan that took up to 200 times longer to acquire. This pipeline, therefore, enables more robust data extraction from fast time-resolved X-ray tomography, removing a critical stumbling block for this technique.

Contrastive Adversarial Learning for Endomicroscopy Imaging Super-Resolution.

Endomicroscopy is an emerging imaging modality for real-time optical biopsy. One limitation of existing endomicroscopy based on coherent fibre bundles is that the image resolution is intrinsically limited by the number of fibres that can be practically integrated within the small imaging probe. To improve the image resolution, Super-Resolution (SR) techniques combined with image priors can enhance the clinical utility of endomicroscopy whereas existing SR algorithms suffer from the lack of explicit guidance from ground truth high-resolution (HR) images. In this paper, we propose an unsupervised SR pipeline to allow stable offline and kernel-generic learning. Our method takes advantage of both internal statistics and external cross-modality priors. To improve the joint learning process, we present a Sharpness-aware Contrastive Generative Adversarial Network (SCGAN) with two dedicated modules, a sharpness-aware generator and a contrastive-learning discriminator. In the generator, an auxiliary task of sharpness discrimination is formulated to facilitate internal learning by considering the rankings of training instances in various sharpness levels. In the discriminator, we design a contrastive-learning module to mitigate the ill-posed nature of SR tasks via constraints from both positive and negative images. Experiments on multiple datasets demonstrate that SCGAN reduces the performance gap between previous unsupervised approaches and the upper bounds defined in supervised settings by more than 50%, delivering a new state-of-the-art performance score for endomicroscopy super-resolution. Further application on a realistic Voronoi-based pCLE downsampling kernel proves that SCGAN attains PSNR of 35.851dB, improving 5.23dB compared with the traditional Delaunay interpolation.

Unpaired image super-resolution using a lightweight invertible neural network

Unpaired image super-resolution (SR) has recently attracted considerable attention in the unsupervised SR community. In contrast to supervised SR, existing unpaired SR methods inevitably resort to the generative adversarial network (GAN) to explore data distribution on the given HR and unpaired LR dataset. Nevertheless, predominant strategies often strive for sophisticated network structures or training pipelines, making them intractable to apply in real-world scenarios. In this work, a lightweight invertible neural network (INN) is proposed for unpaired SR to alleviate this limitation. Specifically, we regard image degradation and SR as a pair of mutually-inverse tasks and replace the two generators in one-stage GAN with INN. Due to the information lossless nature of INN, it is impossible to generate noise in vain during image degradation. We thus design a simple noise injection network to induce realistic noise, thereby simulating real LR images. To further maintain the stability and realism of the noise, we propose to extract the noise prior from the real-world LR image. With extracted noise prior as input, our noise injection network can narrow the gap between the generated noise and the real one, thereby encouraging the degraded images to match the real-world LR domain. Extensive experiments demonstrate that our method achieves comparable performance with other SOTA methods in quantitative and qualitative evaluations while enjoying faster speed and much smaller parameters.

Unsupervised Super Resolution Network for Hyperspectral Histologic Imaging.

Hyperspectral imaging (HSI) has many advantages in microscopic applications, including high sensitivity and specificity for cancer detection on histological slides. However, acquiring hyperspectral images of a whole slide with a high image resolution and a high image quality can take a long scanning time and require a very large data storage. One potential solution is to acquire and save low-resolution hyperspectral images and reconstruct the high-resolution ones only when needed. The purpose of this study is to develop a simple yet effective unsupervised super resolution network for hyperspectral histologic imaging with the guidance of RGB digital histology images. High-resolution hyperspectral images of hemoxylin & eosin (H&E) stained slides were obtained at 10× magnification and down-sampled 2×, 4×, and 5× to generate low-resolution hyperspectral data. High-resolution digital histologic RGB images of the same field of view (FOV) were cropped and registered to the corresponding high-resolution hyperspectral images. A neural network based on a modified U-Net architecture, which takes the low-resolution hyperspectral images and high-resolution RGB images as inputs, was trained with unsupervised methods to output high-resolution hyperspectral data. The generated high-resolution hyperspectral images have similar spectral signatures and improved image contrast than the original high-resolution hyperspectral images, which indicates that the super resolution network with RGB guidance can improve the image quality. The proposed method can reduce the acquisition time and save storage space taken up by hyperspectral images without compromising image quality, which will potentially promote the use of hyperspectral imaging technology in digital pathology and many other clinical applications.

Unsupervised arterial spin labeling image superresolution via multiscale generative adversarial network.

Arterial spin labeling (ASL) magnetic resonance imaging (MRI) is an advanced noninvasive imaging technology that can measure cerebral blood flow (CBF) quantitatively without a contrast agent injection or radiation exposure. However, because of the weak labeling, conventional ASL images usually suffer from low signal-to-noise ratio (SNR), poor spatial resolution, and long acquisition time. Therefore, a method that can simultaneously improve the spatial resolution and SNR isneeded. In this work, we proposed an unsupervised superresolution (SR) method to improve ASL image resolution based on a pyramid of generative adversarial networks (GAN). Through layer-by-layer training, the generators can learn features from the coarsest to the finest. The last layer's generator that contains fine details and textures was used to generate the final SR ASL images. In our proposed framework, the corresponding T1-weighted MR image was supplied as a second-channel input of the generators to provide high-resolution prior information. In addition, a low-pass-filter loss term was included to suppress the noise of the original ASL images. To evaluate the performance of the proposed framework, a simulation study and two real-patient experiments based on the in vivo datasets obtained from three healthy subjects on a 3T MR scanner were conducted, regarding the low-resolution (LR) to normal-resolution (NR) and the NR-to-SR tasks. The proposed method was compared to the nearest neighbor interpolation, trilinear interpolation, third-order B-splines interpolation methods, and deep image prior (DIP) with the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) as the quantification metrics. The averaged ASL images acquired with 44 min acquisition time were used as the ground truth for real-patient LR-to-NR study. The ablation studies of low-pass-filter loss term and T1-weighted MR image were performed based on simulation data. For the simulation study, results show that the proposed method achieved significantly higher PSNR ( -value 0.05) and SSIM ( -value 0.05) than the nearest neighbor interpolation, trilinear interpolation, third-order B-splines interpolation, and DIP methods. For the real-patient LR-to-NR experiment, results show that the proposed method can generate high-quality SR ASL images with clearer structure boundaries and low noise levels and has the highest mean PSNR and SSIM. For real-patient NR-to-SR tasks, the structure of the results using the proposed method is sharper and clearer, which are the most similar to the structure of the reference 44 min acquisition image than other methods. The proposed method also shows the ability to remove artifacts in the NR image while superresolution. The ablation study verified that the low-pass-filter loss term and T1-weighted MR image are necessary for the proposedmethod. The proposed unsupervised multiscale GAN framework can simultaneously improve spatial resolution and reduce image noise. Experiment results from simulation data and three healthy subjects show that the proposed method achieves better performance than the nearest neighbor interpolation, the trilinear interpolation, the third-order B-splines interpolation, and DIP methods.

Wasserstein Patch Prior for Image Superresolution

Many recent superresolution methods are based on supervised learning. That means, that they require a large database of pairs of high- and low-resolution images as training data. However, for many applications, acquiring registered pairs of high and low resolution data or even imaging a large area with a high resolution is unrealistic. To overcome this problem, we introduce a Wasserstein patch prior for unsupervised superresolution of two- and three-dimensional images. In addition to the low-resolution observation, our method only requires one, possibly small, reference image which has a similar patch distribution as the high resolution ground truth. This assumption can e.g. be fulfilled when working with texture images or images of homogeneous material microstructures. The proposed regularizer penalizes the Wasserstein-2-distance of the patch distributions within the reconstruction and the reference image at different scales. We demonstrate the performance of the proposed method by applying it to two- and three-dimensional images of materials' microstructures.

Unsupervised Real-World Image Super-Resolution via Dual Synthetic-to-Realistic and Realistic-to-Synthetic Translations

Due to the challenges of collecting paired low-resolution (LR) and high-resolution (HR) images in real-world scenarios, most existing deep convolutional neural network (CNN)-based single image super-resolution (SR) models are trained with artificially synthesized LR-HR image pairs. However, the domain gap between the synthetic data for model training and the realistic data for testing degrades SR performance significantly, which discourages the application of SR models in practice. One possible solution is to learn from unpaired real-world LR and HR images for their accessibility. Predominant strategies are mainly based on unsupervised domain translation. Despite great advances, there are still noticeable domain gaps between the realistic-like/synthetic-like images generated by unpaired translation and the true realistic/synthetic ones. To address this problem, this letter proposes an effective unsupervised SR framework based on dual synthetic-to-realistic and realistic-to-synthetic translations, namely DTSR. Specifically, to bridge the domain gap between testing and training data, the SR model is optimized using HR images and their realistic-like LR counterparts produced by the synthetic-to-realistic translation. In turn, we propose to narrow the domain gap further via applying the realistic-to-synthetic translation to realistic LR images prior to super-resolving, which also makes the SR model super-resolve simpler examples in testing relative to model training. Moreover, focal frequency and bilateral filtering losses are particularly introduced into DTSR for better details restoration and artifacts suppression. Extensive experiments show that our DTSR outperforms several state-of-the-art models in terms of both quantitative and qualitative comparisons.

Super Resolution of Magnetic Resonance Images.

In this work, novel denoising and super resolution (SR) approaches for magnetic resonance (MR) images are addressed, and are integrated in a unified framework, which do not require example low resolution (LR)/high resolution (HR)/cross-modality/noise-free images and prior information of noise–noise variance. The proposed method categorizes the patches as either smooth or textured and then denoises them by deploying different denoising strategies for efficient denoising. The denoising algorithm is integrated into the SR approach, which uses a gradient profile-based constraint in a sparse representation-based framework to improve the resolution of MR images with reduced smearing of image details. This constraint regularizes the estimation of HR images such that the estimated HR image has gradient profiles similar to the gradient profiles of the original HR image. For this, the gradient profile sharpness (GPS) values of an unknown HR image are estimated using an approximated piece-wise linear relation among GPS values of LR and upsampled LR images. The experiments are performed on three different publicly available datasets. The proposed SR approach outperforms the existing unsupervised SR approach addressed for real MR images that exploits low rank and total variation (LRTV) regularization, by an average peak signal to noise ratio (PSNR) of 0.73 dB and 0.38 dB for upsampling factors 2 and 3, respectively. For the super resolution of noisy real MR images (degraded with 2% noise), the proposed approach outperforms the LRTV approach by an average PSNR of 0.54 dB and 0.46 dB for upsampling factors 2 and 3, respectively. The qualitative analysis is shown for real MR images from healthy subjects and subjects with Alzheimer’s disease and structural deformity, i.e., cavernoma.

Enhanced image prior for unsupervised remoting sensing super-resolution

Numerous approaches based on training low-high resolution image pairs have been proposed to address the super-resolution (SR) task. Despite their success, low-high resolution image pairs are usually difficult to obtain in certain scenarios, and these methods are limited in the actual scene (unknown or non-ideal image acquisition process). In this paper, we proposed a novel unsupervised learning framework, termed Enhanced Image Prior (EIP), which achieves SR tasks without low/high resolution image pairs. We first feed random noise maps into a designed generative adversarial network (GAN) for satellite image SR reconstruction. Then, we convert the reference image to latent space as the enhanced image prior. Finally, we update the input noise in the latent space with a recurrent updating strategy, and further transfer the texture and structured information from the reference image. Results on extensive experiments on the Draper dataset show that EIP achieves significant improvements over state-of-the-art unsupervised SR methods both quantitatively and qualitatively. Our experiments on satellite (SuperView-1) images reveal the potential of the proposed approach in improving the resolution of remote sensing imagery compared with the supervised algorithms. Source code is available at https://github.com/jiaming-wang/EIP.

Zero-Shot Super-Resolution With a Physically-Motivated Downsampling Kernel for Endomicroscopy.

Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth high-resolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for ground-truth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope's irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts.

Direct Unsupervised Super-Resolution Using Generative Adversarial Network (DUS-GAN) for Real-World Data.

The deep learning models for the Single Image Super-Resolution (SISR) task have found success in recent years. However, one of the prime limitations of existing deep learning-based SISR approaches is that they need supervised training. Specifically, the Low-Resolution (LR) images are obtained through known degradation (for instance, bicubic downsampling) from the High-Resolution (HR) images to provide supervised data as an LR-HR pair. Such training results in a domain shift of learnt models when real-world data is provided with multiple degradation factors not present in the training set. To address this challenge, we propose an unsupervised approach for the SISR task using Generative Adversarial Network (GAN), which we refer to hereafter as DUS-GAN. The novel design of the proposed method accomplishes the SR task without degradation estimation of real-world LR data. In addition, a new human perception-based quality assessment loss, i.e., Mean Opinion Score (MOS), has also been introduced to boost the perceptual quality of SR results. The pertinence of the proposed method is validated with numerous experiments on different reference-based (i.e., NTIRE Real-world SR Challenge validation dataset) and no-reference based (i.e., NTIRE Real-world SR Challenge Track-1 and Track-2) testing datasets. The experimental analysis demonstrates committed improvement from the proposed method over the other state-of-the-art unsupervised SR approaches, both in terms of subjective and quantitative evaluations on different reference metrics (i.e., LPIPS, PI-RMSE graph) and no-reference quality measures such as NIQE, BRISQUE and PIQE. We also provide the implementation of the proposed approach (https://github.com/kalpeshjp89/DUSGAN) to support reproducible research.

A Multi-Degradation Aided Method for Unsupervised Remote Sensing Image Super Resolution With Convolution Neural Networks

In remote sensing, it is desirable to improve image resolution by using the image super-resolution (SR) technique. However, there are two challenges: the first one is that high-resolution (HR) images are insufficient or unavailable; another one is that the single degradation model such as bicubic (BIC) cannot super-resolve favorable images in the real world. To address the above two problems, this article presents a multi-degradation, unsupervised SR method based on deep learning. This framework consists of a degrader <inline-formula> <tex-math notation="LaTeX">$ {D}$ </tex-math></inline-formula> to fit the image degradation model and a generator <inline-formula> <tex-math notation="LaTeX">$ {G}$ </tex-math></inline-formula> to generate SR image. By introducing <inline-formula> <tex-math notation="LaTeX">$ {D}$ </tex-math></inline-formula>, calculating the loss function between SR image and HR image as supervised SR methods did can be converted into calculating loss between low resolution (LR) image and image degraded by SR image, thereby realizing unsupervised learning. Experiments on several degradation models show that our method renders the state-of-the-art results compared with existing unsupervised SR methods, and achieves competitive results in contrast with supervised SR methods. Moreover, for real remote sensing images obtained by the Jilin-1 satellite, our method obtained more plausible results visually, which demonstrate the potential in real-world applications.

VEHICLE DETECTION IN REMOTE SENSING IMAGES USING DEEP NEURAL NETWORKS AND MULTI-TASK LEARNING

Abstract. Vehicle detection in remote sensing image has been attracting remarkable attention over past years for its applications in traffic, security, military, and surveillance fields. Due to the stunning success of deep learning techniques in object detection community, we consider to utilize CNNs for vehicle detection task in remote sensing image. Specifically, we take advantage of deep residual network, multi-scale feature fusion, hard example mining and homography augmentation to realize vehicle detection, which almost integrates all the advanced techniques in deep learning community. Furthermore, we simultaneously address super-resolution (SR) and detection problems of low-resolution (LR) image in an end-to-end manner. In consideration of the absence of paired low-/highresolution data which are generally time-consuming and cumbersome to collect, we leverage generative adversarial network (GAN) for unsupervised SR. Detection loss is back-propagated to SR generator to boost detection performance. We conduct experiments on representative benchmark datasets and demonstrate that our model yields significant improvements over state-of-the-art methods in deep learning and remote sensing areas.

Unsupervised Super-Resolution of OCT Images Using Generative Adversarial Network for Improved Age-Related Macular Degeneration Diagnosis

Age-related macular degeneration (AMD) is the leading cause of progressive vision loss in the elderly. Optical coherence tomography (OCT) is a promising diagnostic tool for early detection and management of AMD. However, the speckle noise and low resolution (LR) of the OCT images affect its diagnostic viabilities. Therefore, denoising and super-resolution (SR) techniques present a potential solution to improve the quality of the OCT images. Recent methods rely on example-based approaches that require paired LR and high resolution (HR) images for training. However, the large scale acquisition of paired LR-HR images presents pertinent challenges in clinical settings. Therefore, this work proposes an unsupervised framework using the generative adversarial network (GAN) to perform fast and reliable SR without the requirement of aligned LR-HR pairs. We use adversarial learning with cycle consistency and identity mapping priors to preserve the spatial correlation, color and texture details in the generated clean HR images. Experimental results on clinical-grade OCT images show that the proposed method outperforms the existing methods both in terms of SR performance and computational time. Improved classification accuracy of 96.54% is obtained when the generated images are used for automated AMD diagnosis. The enhanced generalizability and faithful reconstruction attributes make the proposed method suitable for assisting ophthalmologists in better diagnosis and treatment planning.

Achieving Higher Resolution Lake Area from Remote Sensing Images Through an Unsupervised Deep Learning Super-Resolution Method

Lakes have been identified as an important indicator of climate change and a finer lake area can better reflect the changes. In this paper, we propose an effective unsupervised deep gradient network (UDGN) to generate a higher resolution lake area from remote sensing images. By exploiting the power of deep learning, UDGN models the internal recurrence of information inside the single image and its corresponding gradient map to generate images with higher spatial resolution. The gradient map is derived from the input image to provide important geographical information. Since the training samples are only extracted from the input image, UDGN can adapt to different settings per image. Based on the superior adaptability of the UDGN model, two strategies are proposed for super-resolution (SR) mapping of lakes from multispectral remote sensing images. Finally, Landsat 8 and MODIS (moderate-resolution imaging spectroradiometer) images from two study areas on the Tibetan Plateau in China were used to evaluate the performance of UDGN. Compared with four unsupervised SR methods, UDGN obtained the best SR results as well as lake extraction results in terms of both quantitative and visual aspects. The experiments prove that our approach provides a promising way to break through the limitations of median-low resolution remote sensing images in lake change monitoring, and ultimately support finer lake applications.

Deep Learning for Image Super-Resolution: A Survey.

Image Super-Resolution (SR) is an important class of image processing techniqueso enhance the resolution of images and videos in computer vision. Recent years have witnessed remarkable progress of image super-resolution using deep learning techniques. This article aims to provide a comprehensive survey on recent advances of image super-resolution using deep learning approaches. In general, we can roughly group the existing studies of SR techniques into three major categories: supervised SR, unsupervised SR, and domain-specific SR. In addition, we also cover some other important issues, such as publicly available benchmark datasets and performance evaluation metrics. Finally, we conclude this survey by highlighting several future directions and open issues which should be further addressed by the community in the future.

An Efficient Deep Unsupervised Superresolution Model for Remote Sensing Images

Superresolution (SR) has provided an effective solution to the increasing need for high-resolution images in remote sensing applications. Among various SR methods, deep learning-based SR (DLSR) has made a significant breakthrough. However, supervised DLSR methods require a considerable amount of training data, which is hardly available in the remote sensing field. To address this issue, some research works have recently proposed and revealed the capability of deep learning in unsupervised SR. This article presents an efficient unsupervised SR (EUSR) deep learning model using dense skip connections, which boosts the reconstruction performance in parallel with the reduction of computational burden. To do this, several blocks containing densely connected convolutional layers are implemented to increase the depth of the model. Some skip connections also concatenate feature maps of different blocks to enable better SR performance. Moreover, a bottle-neck block abstracts the feature maps in fewer feature maps to remarkably reduce the computational burden. According to our experiments, the proposed EUSR leads to better results than the state-of-the-art DLSR method in terms of reconstruction quality with less computational burden. Furthermore, results indicate that the EUSR is more robust than its rival in dealing with images of different classes and larger sizes.

An Unsupervised Remote Sensing Single-Image Super-Resolution Method Based on Generative Adversarial Network

Image super-resolution (SR) technique can improve the spatial resolution of images without upgrading the imaging system. As a result, SR promotes the development of high resolution (HR) remote sensing image applications. Many remote sensing image SR algorithms based on deep learning have been proposed recently, which can effectively improve the spatial resolution under the constraints of HR images. However, images acquired by remote sensing imaging devices typically have lower resolution. Hence, an insufficient number of HR remote sensing images are available for training deep neural networks. In view of this problem, we propose an unsupervised SR method that does not require HR remote sensing images. The proposed method introduces a generative adversarial network (GAN) that obtains SR images through the generator; then, the SR images are downsampled to train the discriminator with low resolution (LR) images. Our method outperformed several methods in terms of the quality of the obtained SR images as measured by 6 evaluation metrics, which proves the satisfactory performance of the proposed unsupervised method for improving the spatial resolution of remote sensing images.