Low-resolution Images Research Articles

SEN2NAIP: A large-scale dataset for Sentinel-2 Image Super-Resolution

The increasing demand for high spatial resolution in remote sensing has underscored the need for super-resolution (SR) algorithms that can upscale low-resolution (LR) images to high-resolution (HR) ones. To address this, we present SEN2NAIP, a novel and extensive dataset explicitly developed to support SR model training. SEN2NAIP comprises two main components. The first is a set of 2,851 LR-HR image pairs, each covering 1.46 square kilometers. These pairs are produced using LR images from Sentinel-2 (S2) and corresponding HR images from the National Agriculture Imagery Program (NAIP). Using this cross-sensor dataset, we developed a degradation model capable of converting NAIP images to match the characteristics of S2 imagery (S2like). This led to the creation of a second subset, consisting of 35,314 NAIP images and their corresponding S2like counterparts, generated using the degradation model. With the SEN2NAIP dataset, we aim to provide a valuable resource that facilitates the exploration of new techniques for enhancing the spatial resolution of Sentinel-2 imagery.

Diffusion models for super-resolution microscopy: A tutorial

Abstract Diffusion models have emerged as a prominent technique in generative modeling with neural networks, making their mark in tasks like text-to-image translation and super-resolution. In this tutorial, we provide a comprehensive guide to build denoising diffusion probabilistic models (DDPMs) from scratch, with a specific focus on transforming low-resolution microscopy images into their corresponding high-resolution versions in the context of super-resolution microscopy. We provide the necessary theoretical background, the essential mathematical derivations, and a detailed Python code implementation using PyTorch. We discuss the metrics to quantitatively evaluate the model, illustrate the model performance at different noise levels of the input low-resolution images, and briefly discuss how to adapt the tutorial for other applications. The code provided in this tutorial is also available as a Python notebook in the supplementary information.

Enhancing image quality in circular-view photoacoustic tomography using randomized detection points

Abstract Circular-view (circular scan) photoacoustic computed tomography (PACT) with low-density detection points (DPs) is an efficient, high-speed, and inexpensive modality with numerous (pre-) clinical applications. However, as the number of DPs decreases (decrease in A-lines), some unwanted streak artifacts appear in the reconstructed PACT image. Here, we present an approach to address the mentioned challenge and enhance image contrast. In this method, several low-resolution-images (LRIs) are reconstructed by employing a few DPs' data with randomized locations. These LRIs are used in computing an artifact score matrix (ASM) to identify the location of artifacts. Three numerical (two vasculatures and human brain), two experimental (triangle and complex leaf), and an in vivo (a rat brain) studies were conducted to evaluate the efficacy of the proposed method (applying the computed ASM to the final reconstructed photoacoustic image). Our findings show that the proposed method outperforms conventional methods and offers better image quality. The signal-to-noise ratio (SNR) and structural similarity index values (SSIM) of the proposed method are quantitatively 20 dB and 25% better than the conventional method, respectively. Furthermore, compared to the conventional method, the proposed method has an artifact standard deviation that is 50 times lower.

A novel deep unsupervised approach for super-resolution of remote sensing hyperspectral image using gompertz-function convergence war accelerometric-optimization generative adversarial network (GF-CWAO-GAN)

Hyperspectral remote sensing images obtained from cameras are characterized by high-dimensions and low quality, which makes them unfavorable for various analytics purposes. This is due to the presence of visible and invisible frequencies of the reflected light making it poorly reveal the spectral signatures of the image. Visual communication advancement has paved the need for Image Super-Resolution (SR) which recovers high-resolution images from low-resolution images. Several works were carried out earlier on image SR using variants of supervised and unsupervised models that still lack accuracy. In this paper, we propose an unsupervised learning model titled Gompertz Function–based Convergence War Accelerometric Optimization–GAN framework for generating of High-Resolution (HR) images. The framework comprises a pre-processing stage, where the incoming Low-Resolution (LR) image is preprocessed for noise removal by applying Shannon-Gaussian Filter (S-GF). Following is the Gradient Domain Approach based Tone-Mapping (TM). Skew correction is done to remove distortion and maintain original resolution that may change during TM stage. The next stage comprises the boundary and edge enhancement of the resulting preprocessed image generated by the method of Inverse Gradient Mapping (IGM) followed by patch extraction to extract minute low-frequency information from the resulting boundary and edge-enhanced image. The contrast of the enhanced patches is improved by removing blurriness effect. The preprocessed image patches are then fed into the Gompertz Function-based Convergence War Accelerometric Optimization – GAN for feature mapping on the trained SR Image features that are clustered using Krzanowski and Li- Kantorovich Metric-K-Means clustering Algorithm (KL-KM-KMA) for effective generation of SR image. The developed model is validated for both qualitative and quantitative measurements. Comparisons are made with several other state-of -the-art methods for accuracy of 98.05%, precision of 97.98%, inception score of 8.71, Fréchet Inception Distance of 36.4 with reduced clustering and training time proving the efficiency of the proposed model.

Comparative study of generalized-sampling-theorem-based digital super-resolution for 2D data

The size of pixels in a digital recording device, such as a CCD array, limits the spatial resolution in images obtained by an optical imaging system, thereby degrading the image quality. Digital super-resolution (DSR) techniques are used to reconstruct a high-resolution (HR) image from multiple sub-pixel-shifted low-resolution images in order to improve the image quality. In this article, we formulate a mathematical framework for DSR using the generalized sampling theorem (GST). The GST-based DSR method’s performance is evaluated by comparing it to existing resolution enhancement methods on the basis of evaluation metrics like percentage mean square error (%MSE), structural similarity index measure (SSIM), and peak signal-to-noise ratio (PSNR). The GST DSR method exhibits an overall superior quality image reconstruction based on quantitative analysis (with a near zero %MSE, SSIM around one, and improved PSNR in dB) and qualitative comparison. The robustness of the GST DSR method is further demonstrated in the presence of frame-to-frame shift estimation error using %MSE and SSIM by comparing it with multi-frame interpolation approaches.

DESRGAN: Detail-enhanced generative adversarial networks for small sample single image super-resolution

Image super-resolution involves reconstructing a blurry, low-resolution image with limited information into a clear, high-resolution image containing more detailed information. The images generated by super-resolution reconstruction can enhance the performance of downstream computer vision tasks, and hold wide application prospects in fields such as industrial fault detection, plant phenotype parameter extraction, medical imaging, and more. High-frequency components in images, such as edges and texture details, typically require more attention. However, when the training samples are limited, effectively recovering clear high-frequency details of images becomes highly challenging. Therefore, this paper proposes a single-image super-resolution method based on generative adversarial networks, named DESRGAN. Compared to existing methods, DESRGAN achieves better reconstruction of image details even with a limited number of training samples. DESRGAN introduces several key innovations: a shallow generator structure to address overfitting issues in small sample scenarios, a dual-stream feature extraction network with dilated convolutions to capture multi-scale contextual information and expand the receptive field, and an artifact loss designed to eliminate artifacts and preserve the true high-frequency details of the super-resolved images. Extensive ablation experiments and comparative studies with multiple state-of-the-art models are conducted on two small sample datasets, "Root" and "Leaves," as well as five publicly available datasets. The results demonstrate that the proposed DESRGAN achieves superior performance in small sample single image super-resolution tasks, with improvements of 1.39 dB in PSNR and 0.013 in SSIM. The generated high-resolution images exhibit clear texture and edge structures, presenting favorable subjective visual effects. Moreover, the model displays strong generalization capabilities.

Learning Many-to-Many Mapping for Unpaired Real-World Image Super-Resolution and Downscaling.

Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a two-stage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional many-to-many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.

Towards Lightweight Super-Resolution With Dual Regression Learning.

Deep neural networks have exhibited remarkable performance in image super-resolution (SR) tasks by learning a mapping from low-resolution (LR) images to high-resolution (HR) images. However, the SR problem is typically an ill-posed problem and existing methods would come with several limitations. First, the possible mapping space of SR can be extremely large since there may exist many different HR images that can be super-resolved from the same LR image. As a result, it is hard to directly learn a promising SR mapping from such a large space. Second, it is often inevitable to develop very large models with extremely high computational cost to yield promising SR performance. In practice, one can use model compression techniques to obtain compact models by reducing model redundancy. Nevertheless, it is hard for existing model compression methods to accurately identify the redundant components due to the extremely large SR mapping space. To alleviate the first challenge, we propose a dual regression learning scheme to reduce the space of possible SR mappings. Specifically, in addition to the mapping from LR to HR images, we learn an additional dual regression mapping to estimate the downsampling kernel and reconstruct LR images. In this way, the dual mapping acts as a constraint to reduce the space of possible mappings. To address the second challenge, we propose a dual regression compression (DRC) method to reduce model redundancy in both layer-level and channel-level based on channel pruning. Specifically, we first develop a channel number search method that minimizes the dual regression loss to determine the redundancy of each layer. Given the searched channel numbers, we further exploit the dual regression manner to evaluate the importance of channels and prune the redundant ones. Extensive experiments show the effectiveness of our method in obtaining accurate and efficient SR models.

Enhancing image quality using super-resolution residual network for small, blurry images

<span lang="EN-US">In the background, when low-resolution images are utilized, image identification tasks are frequently hampered. By employing the residual network super-resolution framework, super-resolution techniques are used to enhance image quality, specifically in the detection and identification of small and blurry objects. Improving resolution, decreasing blur, and enhancing object detail are the main goals of the suggested approach. The novelty of this research resides in its application of the activation exponential linear unit (ELU) to the super-resolution residual network (SR-ResNet) framework, which has been demonstrated to enhance image sharpness. The experimental findings demonstrate a substantial enhancement in the quality of the images, as evidenced by the training data's structural similarity index (SSIM) of 0.9989 and peak signal-to-noise ratio (PSNR) of 91.8455. Furthermore, the validation data demonstrated SSIM 0.9990 and PSNR 92.5520. The results of this study indicate that the implementation of SR-ResNet significantly enhances the capability of the detection system to detect and classify diminutive and opaque entities precisely. The expected and projected enhancement in image quality significantly influences image processing, especially in situations where accuracy and object differentiation are vital.</span>

Google, SEO and helpful content: How artificial intelligence can be helpful for e-commerce websites

This paper explores the impact of artificial intelligence (AI) on search engine optimisation (SEO) practices within a marketing agency setting, with a particular focus on quality, topical authority and relevance over quantity. By leveraging AI-powered content creation tools, the study explores the effectiveness of generating various SEO elements for enhancing the visibility and ranking of product pages. Using models such as OpenAI’s text-davinci-003, the study demonstrates the generation of SEO elements tailored to specific product details, brand names and website URLs. Through examples of fitness equipment, the study showcases the creation of SEO-friendly content, enhancing the discoverability of product pages and user engagement. Furthermore, the study extends its focus beyond textual content to include the generation of image alt tags, identifying low-resolution images and implementing contextual internal linking using AI techniques. The study explores methodologies for generating alt tags for images based on image URLs, identifying low-resolution images from a list of URLs, and recommending internal links based on content similarity and topical relevance. The study also describes practical applications of AI in various SEO optimisation tasks, emphasising the role of AI-driven solutions in enhancing website crawlability, user experience and overall SEO performance, thereby contributing to improved visibility, traffic and conversion rates. By addressing key SEO challenges and leveraging AI technologies, marketing agencies can optimise their strategies for better search engine visibility, ultimately driving growth and competitiveness in the digital landscape.

NExpR: Neural Explicit Representation for fast arbitrary-scale medical image super-resolution

Medical images often require rescaling to various spatial resolutions to ensure interpretations at different levels. Conventional deep learning-based image super-resolution (SR) enhances the fixed-scale resolution. Implicit neural representation (INR) is a promising way of achieving arbitrary-scale image SR. However, existing INR-based methods require the repeated execution of the neural network (NN), which is slow and inefficient. In this paper, we present Neural Explicit Representation (NExpR) for fast arbitrary-scale medical image SR. Our algorithm represents an image with an explicit analytical function, whose input is the low-resolution image and output is the parameterization of the analytical function. After obtaining the analytical representation through a single NN inference, SR images of arbitrary scales can be derived by evaluating the explicit functions at desired coordinates. Because of the analytical explicit representation, NExpR is significantly faster than INR-based methods. In addition to speed, our method achieves on-par or better image quality than other strong competitors. Extensive experiments on Magnetic Resonance Imaging (MRI) datasets, including ProstateX, fastMRI, and our in-house clinical prostate dataset, as well as the Computerized Tomography (CT) dataset, specifically the Medical Segmentation Decathlon (MSD) liver dataset, demonstrate the superiority of our method. Our method reduces the rescaling time from the order of 1 ms to the order of 0.01 ms, achieving an over 100× speedup without losing the image quality. Code is available at https://github.com/Calvin-Pang/NExpR.

HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks

The objective of hyperspectral pansharpening is to fuse low-resolution hyperspectral images (LR-HSI) with corresponding panchromatic (PAN) images to generate high-resolution hyperspectral images (HR-HSI). Despite advancements in hyperspectral (HS) pansharpening using deep learning, the rich spectral details and large data volume of HS images place higher demands on models for effective spectral extraction and processing. In this paper, we present HyperGAN, a hyperspectral image fusion approach based on Generative Adversarial Networks. Unlike previous methods that deepen the network to capture spectral information, HyperGAN widens the structure with a Wide Block for multi-scale learning, effectively capturing global and local details from upsampled HSI and PAN images. While LR-HSI provides rich spectral data, PAN images offer spatial information. We introduce the Efficient Spatial and Channel Attention Module (ESCA) to integrate these features and add an energy-based discriminator to enhance model performance by learning directly from the Ground Truth (GT), improving fused image quality. We validated our method on various scenes, including the Pavia Center, Eastern Tianshan, and Chikusei. Results show that HyperGAN outperforms state-of-the-art methods in visual and quantitative evaluations.

Resolution-dependent MRI-to-CT translation for orthotopic breast cancer models using deep learning

Objective.This study aims to investigate the feasibility of utilizing generative adversarial networks (GANs) to synthesize high-fidelity computed tomography (CT) images from lower-resolution MR images. The goal is to reduce patient exposure to ionizing radiation while maintaining treatment accuracy and accelerating MR image acquisition. The primary focus is to determine the extent to which low-resolution MR images can be utilized to generate high-quality CT images through a systematic study of spatial resolution-dependent magnetic resonance imaging (MRI)-to-CT image conversion.Approach.Paired MRI-CT images were acquired from healthy control and tumor models, generated by injecting MDA-MB-231 and 4T1 tumor cells into the mammary fat pad of nude and BALB/c mice to ensure model diversification. To explore various MRI resolutions, we downscaled the highest-resolution MR image into three lower resolutions. Using a customized U-Net model, we automated region of interest masking for both MRI and CT modalities with precise alignment, achieved through three-dimensional affine paired MRI-CT registrations. Then our customized models, Nested U-Net GAN and Attention U-Net GAN, were employed to translate low-resolution MR images into high-resolution CT images, followed by evaluation with separate testing datasets.Main Results.Our approach successfully generated high-quality CT images (0.142mm2) from both lower-resolution (0.282mm2) and higher-resolution (0.142mm2) MR images, with no statistically significant differences between them, effectively doubling the speed of MR image acquisition. Our customized GANs successfully preserved anatomical details, addressing the typical loss issue seen in other MRI-CT translation techniques across all resolutions of MR image inputs.Significance.This study demonstrates the potential of using low-resolution MR images to generate high-quality CT images, thereby reducing radiation exposure and expediting MRI acquisition while maintaining accuracy for radiotherapy.

Adding Noise to Super-Resolution Training Set: Method to Denoise Super Resolution for Structure from Motion Preprocessing

The resolution and noise levels of input images directly affect the three-dimensional (3D) structure-from-motion (SfM) reconstruction performance. Conventional super-resolution (SR) methods focus too little on denoising, and latent image noise becomes worse when resolution is improved. This study proposes two SR denoising training algorithms to simultaneously improve resolution and noise: add-noise-before-downsampling and downsample-before-adding-noise. These portable methods preprocess low-resolution training images using real-world noise samples instead of altering the basic neural network. Hence, they concurrently improve resolution while reducing noise for an overall cleaner SfM performance. We applied these methods to the existing SR network: super-resolution convolutional neural network, enhanced deep residual super-resolution, residual channel attention network, and efficient super-resolution transformer, comparing their performances with those of conventional methods. Impressive peak signal-to-noise and structural similarity improvements of 0.12 dB and 0.56 were achieved on the noisy images of Smartphone Image Denoising Dataset, respectively, without altering the network structure. The proposed methods caused a very small loss (<0.01 dB) on clean images. Moreover, using the proposed SR algorithm makes the 3D SfM reconstruction more complete. Upon applying the methods to non-preprocessed and conventionally preprocessed models, the mean projection error was reduced by a maximum of 27% and 4%, respectively, and the number of 3D densified points was improved by 310% and 7%, respectively.

High-resolution optical image reconstruction based on adaptive sparse dictionary

Image spatial resolution reflects the ability of an optical system to capture detailed information about an object. Compared to Low-resolution (LR) images, High-resolution (HR) images contain greater pixel density and textural detail. In the experiments, it is difficult to obtain ideal HR images due to the effects of acquisition equipment and image degradation. To address the problems, this paper proposes a HR optical image reconstruction method based on adaptive sparse dictionary. The experimental show that the area of the modulation transfer function (MTF) curve of the reconstructed image is improved by 16.41%, which represents the increase in the frequency components it contains. Meanwhile the cut-off resolution is improved from 0.2692cy/pix to 0.4018cy/pix. The method in paper achieves good results in both reconstruction efficiency and accuracy. The peak signal-to-noise ratio (PSNR) of the reconstructed image is improved from 20.1650 to 28.0192. The feature similarity (FSIM) and detail retention are above 90%.

Low-Complexity Convolutional Neural Network for Channel Estimation

This paper presents a deep learning algorithm for channel estimation in 5G New Radio (NR). The classical approach that uses neural networks for channel estimation requires more than one stage to obtain the full channel matrix. First, the channel has to be constructed by the received reference signal, and then, the precision is improved. In contrast, to reduce the computational cost, the proposed neural network method generates the channel matrix from the information captured from a few subcarriers along the slot. This information is extrapolated by applying the Least Square technique only on the Demodulation Reference Signal (DMRS). The received DMRS placed in the grid can be seen as a 2D low-resolution image and it is processed to generate the full channel matrix. To reduce complexity in the hardware implementation, the convolutional neural network (CNN) structure is selected. This solution is analyzed comparing the Mean Square Error (MSE) and the computational cost with other deep learning-based channel estimation, as well as the traditional channel estimation methods. It is demonstrated that the proposed neural network delivers substantial complexity savings and favorable error performance. It reduces the computational cost by an order of magnitude, and it has a maximum error discrepancy of 0.018 at 5 dB compared to Minimum Mean Square Error (MMSE) channel estimation.

Full-Aperture Reflective Remote Fourier Ptychography with Sample Matching

Fourier ptychography (FP) can break through the limitations of existing optical systems with a single aperture and realize large field-of-view (FOV) and high-resolution (HR) imaging simultaneously by aperture synthesis in the frequency domain. The method has potential applications for remote sensing and space-based imaging. However, the aperture stop of the imaging system was generally set to be much smaller than the system with an adjustable diaphragm, so it failed to make full use of the imaging capability of the system. In this paper, a reflective remote FP with full aperture is proposed, and the optical aperture of the camera is set to be the maximum according to the sample-matching condition, which can further improve the imaging resolution by exploring the whole capability of the system. Firstly, the physical model of the remote FP is established using oblique illumination of a convergent spherical wave. Then, the sampling characteristics of the low-resolution (LR) intensity image are analyzed. Assuming diffraction-limited imaging, the size of the aperture of the optical system needs to match the sampling of the detector. An experimental setup with an imaging distance of 2.4 m is built, and a series of LR images is collected by moving the camera for the diffused samples, including the USAF resolution test target and the banknote, where the diameter of the single aperture is set to the maximum to match the size of the CCD pixel under the practical minimum F# of the camera of 2.8. The high-resolution image is reconstructed by applying the iterative phase retrieval algorithm. The experimental results show that the reconstructed resolution is improved to 2.5×. This verifies that remote FP with full aperture can effectively improve the imaging resolution using only the present single-aperture optical system.

HifiDiff: High-fidelity diffusion model for face hallucination from tiny non-frontal faces

Obtaining a high-quality frontal facial image from a low-resolution (LR) non-frontal facial image is crucial for many facial analysis tasks. Recently, diffusion models (DMs) have made impressive progress in near-frontal face super-resolution. However, when faced with non-frontal LR faces, the existing DMs exhibit poor identity preservation and facial detail fidelity. In this paper, we present a novel high-fidelity DM named HifiDiff for simultaneously super-resolving and frontalizing tiny non-frontal facial images. It consists of a two-stage pipeline: facial preview and facial refinement. In the first stage, we pretrain a coarse restoration module to obtain a coarse high-resolution (HR) frontal face, which serves as a superior constraint condition to enhance the ability to solve complex inverse transform issues. In the second stage, we leverage the strong generation capabilities of the latent DM to refine the facial details. Specifically, we design a two-pathway control structure that consists of a facial prior guidance (FPG) module and an identity consistency (IDC) module to control the denoising process. FPG encodes multilevel features derived from latent coarse HR frontal faces and employs hybrid cross-attention to capture their intrinsic correlations with the denoiser features, thereby improving the fidelity of the facial details. IDC utilizes contrastive learning to extract high-level semantic identity-representing features to constrain the denoiser, thereby maintaining the fidelity of facial identities. Extensive experiments demonstrate that our HifiDiff produces both high-fidelity and realistic HR frontal facial images, surpassing other state-of-the-art methods in qualitative and quantitative analyses, as well as in downstream facial recognition tasks.

Super-resolution reconstruction of light field image based on dynamic adjustment of attention mechanism

The light field imaging has the problem of mutual restriction between spatial resolution and angular resolution. In this paper, we propose a new blind network based on dynamic adjustment of attention mechanism, and the network can reconstruct the low-resolution images with the multiple blur kernels and noise levels to realize super-resolution reconstruction of the light field images. Firstly, the original sub-aperture images array with low-resolution is used to estimate the blur kernel and noise level as auxiliary information. Then, the parameter features are extracted by the dynamic adjustment module, and the degradation representation is combined with the image features to adapt to various blur kernels and noises. After that, the feature information of the light field image is calculated by the distg-block attention block, and a finer feature map can be obtained for subsequent network learning. Finally, the feature map is up-sampled to obtain a sub-aperture image array. The experimental results show that the proposed method can improve the spatial resolution of the light field images by four times, and obtain clearer light field images with more details.

Low-resolution face recognition: Review, challenges and research directions

Low-resolution face recognition (LRFR) is an active research area as it is widely used in forensics and surveillance systems. A lot of effort has been put into improving the performance of the system since its inception. Recent deep neural network models have demonstrated outstanding face recognition performance on various face data sets with challenges like variations in pose, illumination, and occlusion and surpassed the performance of humans in these tasks. But, the accuracy of the LRFR method is still a problem. There is no fixed definition for considering any image as a low-resolution (LR) image. Most of the researchers have considered the image below 32 × 32 as a low-resolution image. This paper discusses various methods and algorithms used in improving the performance of low-resolution face recognition (LRFR). We have presented a thorough study of all the processes included in face recognition tasks including face detection, feature mapping, super-resolution, and face recognition. The study includes methodology along with the dataset and performance measures. We have also summarized the study of different datasets used in LRFR along with various source codes used to perform experimentation on LRFR. We have also presented a study in terms of accuracy of different LRFR methods on different dataset. Finally, challenges and research directions are presented to further carry out the LRFR research.