Blind Super-resolution Research Articles

Self-Supervised Deep Blind Video Super-Resolution.

Existing deep learning-based video super-resolution (SR) methods usually depend on the supervised learning approach, where the training data is usually generated by the blurring operation with known or predefined kernels (e.g., Bicubic kernel) followed by a decimation operation. However, this does not hold for real applications as the degradation process is complex and cannot be approximated by these idea cases well. Moreover, obtaining high-resolution (HR) videos and the corresponding low-resolution (LR) ones in real-world scenarios is difficult. To overcome these problems, we propose a self-supervised learning method to solve the blind video SR problem, which simultaneously estimates blur kernels and HR videos from the LR videos. As directly using LR videos as supervision usually leads to trivial solutions, we develop a simple and effective method to generate auxiliary paired data from original LR videos according to the image formation of video SR, so that the networks can be better constrained by the generated paired data for both blur kernel estimation and latent HR video restoration. In addition, we introduce an optical flow estimation module to exploit the information from adjacent frames for HR video restoration. Experiments show that our method performs favorably against state-of-the-art ones on benchmarks and real-world videos.

Meta-learning based blind image super-resolution approach to different degradations

Although recent studies on blind single image super-resolution (SISR) have achieved significant success, most of them typically require supervised training on synthetic low resolution (LR)-high resolution (HR) paired images. This leads to re-training necessity for different degradations and restricted applications in real-world scenarios with unfavorable inputs. In this paper, we propose an unsupervised blind SISR method with input underlying different degradations, named different degradations blind super-resolution (DDSR). It formulates a Gaussian modeling on blur degradation and employs a meta-learning framework for solving different image degradations. Specifically, a neural network-based kernel generator is optimized by learning from random kernel samples, referred to as random kernel learning. This operation provides effective initialization for blur degradation optimization. At the same time, a meta-learning framework is proposed to resolve multiple degradation modelings on the basis of alternative optimization between blur degradation and image restoration, respectively. Differing from the pre-trained deep-learning methods, the proposed DDSR is implemented in a plug-and-play manner, and is capable of restoring HR image from unfavorable LR input with degradations such as partial coverage, noise addition, and darkening. Extensive simulations illustrate the superior performance of the proposed DDSR approach compared to the state-of-the-arts on public datasets with comparable memory load and time consumption, yet exhibiting better application flexibility and convenience, and significantly better generalization ability towards multiple degradations. Our code is available at https://github.com/XYLGroup/DDSR.

Generation diffusion degradation: Simple and efficient design for blind super-resolution

In the domain of computer vision, blind super-resolution is a key area focused on generating high-resolution images with enhanced visual quality from low-resolution counterparts affected by indeterminate degradation factors. This area is primarily advanced through self-supervised learning techniques utilizing GANs. Despite their prominence, GAN-based methods encounter challenges including unstable training dynamics and limited diversity, compounded by the intricate necessity to configure degradation models to mimic various blur effects and noise types. Lately, denoising diffusion models have shown promising results in image restoration, yet their sampling efficiency constraints impede their deployment in real-time scenarios. This study introduces the Generation Diffusion Degradation (GDD) model, a novel and efficient technique for replicating image degradation by applying random Gaussian noise in a sequential manner via a parametric Markov chain, followed by a progressive reconstruction of the initial image through a U-net-based noise predictor. This method adeptly mirrors the inherent degradation distribution observed in actual degraded images. Furthermore, we present an innovative training strategy that utilizes a composite loss function to train the GDD model, ensuring stable training, improving the authenticity of the generated degraded images, and precisely reflecting the degradation patterns of target images. Extensive experimental analyses underscore the superior performance of the proposed GDD model, both in objective metrics and subjective visual quality. The code is available at https://github.com/lgylab/GDD.

DHBSR: A deep hybrid representation-based network for blind image super resolution

Image super resolution involves enhancing the spatial resolution of low-quality images and improving their visual quality. As in many real-life situations, the image degradation process is unknown, performing the task of image super resolution in a blind manner is of paramount importance. Deep neural networks provide high performances for the task of blind image super resolution, in view of their end-to-end learning capability between the low-resolution images and their ground truth versions. Generally speaking, deep blind image super resolution networks initially estimate the parameters of the image degradation process, such as blurring kernel, and then use them for super-resolving the low-resolution images. In this paper, we develop a novel deep learning-based scheme for the task of blind image super resolution, in which the idea of leveraging the hybrid representations is utilized. Specifically, we employ the deterministic and stochastic representations of the blurring kernel parameters to train a deep blind super resolution network in an effective manner. The results of extensive experiments prove the effectiveness of various ideas used in the development of the proposed deep blind image super resolution network.

A Heterogeneous Group CNN for Image Super-Resolution.

Convolutional neural networks (CNNs) have obtained remarkable performance via deep architectures. However, these CNNs often achieve poor robustness for image super-resolution (SR) under complex scenes. In this article, we present a heterogeneous group SR CNN (HGSRCNN) via leveraging structure information of different types to obtain a high-quality image. Specifically, each heterogeneous group block (HGB) of HGSRCNN uses a heterogeneous architecture containing a symmetric group convolutional block and a complementary convolutional block in a parallel way to enhance the internal and external relations of different channels for facilitating richer low-frequency structure information of different types. To prevent the appearance of obtained redundant features, a refinement block (RB) with signal enhancements in a serial way is designed to filter useless information. To prevent the loss of original information, a multilevel enhancement mechanism guides a CNN to achieve a symmetric architecture for promoting expressive ability of HGSRCNN. Besides, a parallel upsampling mechanism is developed to train a blind SR model. Extensive experiments illustrate that the proposed HGSRCNN has obtained excellent SR performance in terms of both quantitative and qualitative analysis. Codes can be accessed at https://github.com/hellloxiaotian/HGSRCNN.

A boosted degradation representation learning for blind image super-resolution

The significant gap between the assumed and actual degradation model will undoubtedly lead to a serious performance decrease for deep learning based image super-resolution (SR) methods. To solve this shortcoming, this paper presents an enhanced blind image super-resolution (EBSR) network based on unsupervised degradation representation learning. The proposed EBSR mainly is made up of two branching subnetworks. The first subnetwork is a residual-based degradation encoder, which is responsible to learn a high-dimensional abstract degradation representation vector from the input low-resolution (LR) image patches using contrast learning. Another branch is the degradation-aware fusion SR (DAFSR) network that completes the image SR task with the aid of the degradation representation vector learned by the encoder. To obtain an improved SR performance under various degradation settings, the degradation-aware fusion block (DAFB), the core block of DAFSR, has been elaborated, which embeds the degradation model information learned by the encoder into every layer of the network. Additionally, a shallow feature extraction block (SFEB) is constructed for DAFSR to extract global information from the LR image. The proposed EBSR can flexibly adapt to various degradation models. Extensive experimental results on synthetic datasets and real images show that the proposed EBSR is able to achieve a leading reconstruction performance over other blind SR algorithms.

Non-local degradation modeling for spatially adaptive single image super-resolution

Existing methods for single image super-resolution (SISR) model the blur kernel as spatially invariant across the entire image, and are susceptible to the adverse effects of textureless patches. To achieve improved results, adaptive estimation of the degradation kernel is necessary. We explore the synergy of joint global and local degradation modeling for spatially adaptive blind SISR. Our model, named spatially adaptive network for blind super-resolution (SASR), employs a simple encoder to estimate global degradation representations and a decoder to extract local degradation. These two representations are fused with a cross-attention mechanism and applied using spatially adaptive filtering to enhance the local image detail. Specifically, SASR contains two novel features: (1) a non-local degradation modeling with contrastive learning to learn global and local degradation representations, and (2) a non-local spatially adaptive filtering module (SAFM) that incorporates the global degradation and spatial-detail factors to preserve and enhance local details. We demonstrate that SASR can efficiently estimate degradation representations and handle multiple types of degradation. The local representations avoid the detrimental effect of estimating the entire super-resolved image with only one kernel through locally adaptive adjustments. Extensive experiments are performed to quantitatively and qualitatively demonstrate that SASR not only performs favorably for degradation estimation but also leads to state-of-the-art blind SISR performance when compared to alternative approaches.

SwinIBSR: Towards real-world infrared image super-resolution

Traditional infrared image super-resolution (SR) methods often fall short in real-world scenarios, particularly when dealing with images degraded by various forms of noise and artifacts. Addressing this gap, this paper presents SwinIBSR, a blind SR approach optimized for real-world infrared imaging conditions. SwinIBSR introduces a practical degradation model, specifically designed to simulate the complex degradation processes encountered in real-world infrared imaging. Additionally, a mixed training strategy using both infrared and visible light datasets greatly enhances the network’s generalization capabilities, leading to the production of high-quality SR images. Building upon the Transformer-based architecture of SwinIR, SwinIBSR is adept at learning intricate mappings from low-resolution to high-resolution images. Our comprehensive experiments validate the effectiveness of SwinIBSR, showcasing its outstanding performance in processing real-world infrared images and its notable superiority over existing methods regarding visual quality. This work contributes to the field of infrared image SR, offering insights in enhancing SR techniques for practical applications.

Degradation regression with uncertainty for blind super-resolution

Some recent blind super-resolution (SR) efforts focus on designing complex degradation models to better simulate real-world degradations. The paired high-resolution (HR) & low-resolution (LR) samples synthesized by these models can cover a large degradation space, which helps train a robust SR model in real scenarios. However, these diverse synthetic samples may render the SR model degradation-unaware and prevent it from achieving optimal results on LR images with specific degradations. Alternatively, another category of methods is proposed to estimate specific degradations in the given application and then tailor a degradation-aware SR model accordingly. Nonetheless, degradation estimation is an ill-posed problem and accurate estimation is quite challenging. Towards these issues, we propose a probabilistic degradation estimator (PDE) which can predict the degradation as a certain distribution rather than a single point. Specifically, we develop an intersection over union (IoU) based degradation regression loss with uncertainty, which could lead PDE to shrink the possible degradation space of the test LR image. This enables the degradation model to synthesize more degradation-specific training samples and further improve SR performance. In this way, our PDE can alleviate degradation redundancy compared with degradation-unaware methods and is more robust to the degradation estimation error than previous degradation-aware methods. Extensive experiments show that the proposed PDE can help the SR model produce better results on both synthetic and real-world images.

A GPU scheme for multi-secret visual sharing with varied secret dimensions and contrast enhancement using blind super-resolution

Multi-secret visual sharing schemes are essential for secure and efficient sharing of sensitive visual information. However, existing schemes often overlook important considerations such as varied secret dimensions, contrast enhancement, and computational efficiency. This research addresses these challenges by proposing a comprehensive approach that addresses these limitations. Firstly, the scheme takes into account the varied dimensions of secret images encountered in real-world scenarios, allowing flexibility in sharing and reconstructing images of different sizes and aspect ratios. Secondly, the research integrates contrast enhancement techniques, such as blind super-resolution, to improve the visual quality and visibility of shared secret images affected by factors like noise, compression, or low lighting conditions. Lastly, to enhance the computational efficiency, the scheme leverages the power of Graphics Processing Units (GPUs) for parallel computing, enabling faster processing of large-scale image operations. The proposed scheme achieves outstanding results, with a high PSNR of 98.264, a strong NCC value of 0.965, an exceptionally low NAE value of 0.046, and an impressive SSIM value of 0.983. Furthermore, the GPU implementation provides a remarkable overall speedup of 112×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$112\ imes$$\\end{document} for 256×256\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$256 \ imes 256$$\\end{document} color images.

Difficulty-Guided Variant Degradation Learning for Blind Image Super-Resolution.

Recent blind super-resolution (BSR) methods are explored to handle unknown degradations and achieve impressive performance. However, the prevailing assumption in most BSR methods is the spatial invariance of degradation kernels across the entire image, which leads to significant performance declines when faced with spatially variant degradations caused by object motion or defocusing. Additionally, these methods do not account for the human visual system's tendency to focus differently on areas of varying perceptual difficulty, as they uniformly process each pixel during reconstruction. To cope with these issues, we propose a difficulty-guided variant degradation learning network for BSR, named difficulty-guided degradation learning (DDL)-BSR, which explores the relationship between reconstruction difficulty and degradation estimation. Accordingly, the proposed DDL-BSR consists of three customized networks: reconstruction difficulty prediction (RDP), space-variant degradation estimation (SDE), and degradation and difficulty-informed reconstruction (DDR). Specifically, RDP learns the reconstruction difficulty with the proposed reconstruction-distance supervision. Then, SDE is designed to estimate space-variant degradation kernels according to the difficulty map. Finally, both degradation kernels and reconstruction difficulty are fed into DDR, which takes into account such two prior knowledge information to guide super-resolution (SR). Experimental analysis on various synthetic datasets demonstrates that DDL-BSR invariably surpasses state-of-the-art (SOTA) methods, producing SR images with enhanced realism and texture quality. Code is available at https://github.com/JiaWang0704/DDL-BSR.

SBHSR: single-band super-resolution method for hyperspectral images based on blind degradation and fusion of auxiliary band

Despite natural image super-resolution (SR) methods have achieved great success, super-resolution methods for hyperspectral image (HSI) with rich spectral features are still a very challenging task. Furthermore, due to the diversity of HSI captured by different cameras, their degradation conditions vary. Currently available HSI SR methods are mainly based on fixed degradation models, and their performance is severely affected when the actual degradation does not match the assumed degradation model. To address this issue, this paper proposes a single-band hyperspectral image super-resolution (SBHSR) method for HSI. This method assumes that the degradation Gaussian blur kernel parameters of the HSI are unknown, and adapts to various degradation conditions by conducting blind super-resolution on the image. It also combines another band with better spatial structure as an auxiliary band for feature fusion. To address the spectral differences between the super-resolution band and the auxiliary band, we use the Spatial Adaptation Module (SAM) to map the feature distribution, ensuring consistent spectral brightness between the auxiliary band and the super-resolution band. Due to the use of blind degradation super-resolution method, our method is more robust compared to previous HSI SR methods. Additionally, as it is specifically designed for single-band hyperspectral image super-resolution, it has the advantage of being faster and more efficient. Experimental validation on two hyperspectral datasets demonstrates the superiority of our method, as it improves the spatial resolution of the image while preserving the spectral features, and performs better than existing blind super-resolution methods.

Adaptive Blind Super-Resolution Network for Spatial-Specific and Spatial-Agnostic Degradations.

Prior methodologies have disregarded the diversities among distinct degradation types during image reconstruction, employing a uniform network model to handle multiple deteriorations. Nevertheless, we discover that prevalent degradation modalities, including sampling, blurring, and noise, can be roughly categorized into two classes. We classify the first class as spatial-agnostic dominant degradations, less affected by regional changes in image space, such as downsampling and noise degradation. The second class degradation type is intimately associated with the spatial position of the image, such as blurring, and we identify them as spatial-specific dominant degradations. We introduce a dynamic filter network integrating global and local branches to address these two degradation types. This network can greatly alleviate the practical degradation problem. Specifically, the global dynamic filtering layer can perceive the spatial-agnostic dominant degradation in different images by applying weights generated by the attention mechanism to multiple parallel standard convolution kernels, enhancing the network's representation ability. Meanwhile, the local dynamic filtering layer converts feature maps of the image into a spatially specific dynamic filtering operator, which performs spatially specific convolution operations on the image features to handle spatial-specific dominant degradations. By effectively integrating both global and local dynamic filtering operators, our proposed method outperforms state-of-the-art blind super-resolution algorithms in both synthetic and real image datasets.

Bridging Component Learning with Degradation Modelling for Blind Image Super-Resolution

Convolutional Neural Network (CNN)-based image super-resolution (SR) has exhibited impressive success on known degraded low-resolution (LR) images. However, this type of approach is hard to hold its performance in practical scenarios when the degradation process is unknown. Despite existing blind SR methods proposed to solve this problem using blur kernel estimation, the perceptual quality and reconstruction accuracy are still unsatisfactory. In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model. We propose a components decomposition and co-optimization network (CDCN) for blind SR. Firstly, CDCN decomposes the input LR image into structure and detail components in feature space. Then, the mutual collaboration block (MCB) is presented to exploit the relationship between both two components. In this way, the detail component can provide informative features to enrich the structural context and the structure component can carry structural context for better detail revealing via a mutual complementary manner. After that, we present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process. Finally, a multi-scale fusion module followed by an upsampling layer is designed to fuse the structure and detail features and perform SR reconstruction. Empowered by such degradation-based components decomposition, collaboration, and mutual optimization, we can bridge the correlation between component learning and degradation modelling for blind SR, thereby producing SR results with more accurate textures. Extensive experiments on both synthetic SR datasets and real-world images show that the proposed method achieves the state-of-the-art performance compared to existing methods.

Simpler Gradient Methods for Blind Super-Resolution with Lower Iteration Complexity

Simpler Gradient Methods for Blind Super-Resolution with Lower Iteration Complexity

PESiT: Progressive Joint Enhancement and Blind Super-Resolution for Low-Light and Low-Resolution Images Under Total Variation Constraints

PESiT: Progressive Joint Enhancement and Blind Super-Resolution for Low-Light and Low-Resolution Images Under Total Variation Constraints

Lightweight Prompt Learning Implicit Degradation Estimation Network for Blind Super Resolution.

Blind image super-resolution (SR) aims to recover a high-resolution (HR) image from its low-resolution (LR) counterpart under the assumption of unknown degradations. Many existing blind SR methods rely on supervising ground-truth kernels referred to as explicit degradation estimators. However, it is very challenging to obtain the ground-truths for different degradations kernels. Moreover, most of these methods rely on heavy backbone networks, which demand extensive computational resources. Implicit degradation estimators do not require the availability of ground truth kernels, but they see a significant performance gap with the explicit degradation estimators due to such missing information. We present a novel approach that significantly narrows such a gap by means of a lightweight architecture that implicitly learns the degradation kernel with the help of a novel loss component. The kernel is exploited by a learnable Wiener filter that performs efficient deconvolution in the Fourier domain by deriving a closed-form solution. Inspired by prompt-based learning, we also propose a novel degradation-conditioned prompt layer that exploits the estimated kernel to drive the focus on the discriminative contextual information that guides the reconstruction process in recovering the latent HR image. Extensive experiments under different degradation settings demonstrate that our model, named PL-IDENet, yields PSNR and SSIM improvements of more than 0.4dB and 1.3%, and 1.4dB and 4.8% to the best implicit and explicit blind-SR method, respectively. These results are achieved while maintaining a substantially lower number of parameters/FLOPs (i.e., 25% and 68% fewer parameters than best implicit and explicit methods, respectively).

Unsupervised image blind super resolution via real degradation feature learning

AbstractIn recent years, many methods for image super‐resolution (SR) have relied on pairs of low‐resolution (LR) and high‐resolution (HR) images for training, where the degradation process is predefined by bicubic downsampling. While such approaches perform well in standard benchmark tests, they often fail to accurately replicate the complexity of real‐world image degradation. To address this challenge, researchers have proposed the use of unpaired image training to implicitly model the degradation process. However, there is a significant domain gap between the real‐world LR and the synthetic LR images from HR, which severely degrades the SR performance. A novel unsupervised image‐blind super‐resolution method that exploits degradation feature‐based learning for real‐image super‐resolution reconstruction (RDFL) is proposed. Their approach learns the degradation process from HR to LR using a generative adversarial network (GAN) and constrains the data distribution of the synthetic LR with real degraded images. The authors then encode the degraded features into a Transformer‐based SR network for image super‐resolution reconstruction through degradation representation learning. Extensive experiments on both synthetic and real datasets demonstrate the effectiveness and superiority of the RDFL method, which achieves visually pleasing reconstruction results.

KGSR: A kernel guided network for real-world blind super-resolution

In recent years, deep learning-based methods have emerged as dominant players in the field of super-resolution (SR), owing to their exceptional reconstruction performance. The primary driver of their effectiveness lies in their utilization of extensive sets of paired low-resolution and high-resolution images for training deep learning models. This training enables the models to effectively replicate the intricate mapping relationship between low-resolution and high-resolution images. Nevertheless, at present, acquiring a sufficient quantity of such image pairs that satisfy the requirements remains a formidable obstacle. Therefore, in order to break the restriction of limited training sets, self-supervised learning has been introduced to train a model for each low-quality image, without requiring pairwise ground-truths. However, they generally presuppose the generation of low-resolution (LR) images from their high-resolution (HR) counterparts using a pre-defined kernel, such as Bicubic downscaling. Such an assumption is seldom valid for real-world LR images, where degradation processes in practical applications are diverse, intricate, and often undisclosed. Therefore, when the presumed downscaling kernel does not match the actual one, the outcomes of state-of-the-art approaches degrade substantially. In this paper, we introduce KGSR, a kernel-guided network for addressing real-world blind SR, effectively avoiding requiring large training image pairs and transforming the blind image super-resolution problem into a supervised learning and non-blind scenario. Specifically, KGSR trains two networks, namely Upscaling and Downscaling, utilizing only patches extracted from the input test image. On one hand, owing to the cross-scale recurrence property of the SR kernel within a single image, the Downscaling network acquires knowledge of the image-specific degradation process through a generative adversarial network. Consequently, the Downscaling network is capable of generating a downsampled version of the LR test image even when the acquisition process is unknown or less than ideal. Additionally, we employ a dedicated discriminator to compel the Downscaling network to prioritize the characterization of kernel orientations. Conversely, a precise blur kernel has the potential to yield superior performance. Guided by the accurate image-specific SR kernel acquired from the Downscaling network and the downsampled LR input, the Upscaling network is capable of producing a high-quality HR image from the LR input. Within the Upscaling network, we additionally introduce an effective module for harnessing the acquired image-specific SR kernel. KGSR operates as a fully unsupervised approach, yet it can concurrently produce both the image-specific SR kernel and high-quality HR images. Comprehensive experiments conducted on standard benchmarks validate the efficacy of the proposed approach compared to state-of-the-art methodologies. Moreover, the suggested method can deliver visually appealing SR outcomes while exhibiting shorter processing times when applied to real-world LR images.

MDC-Net: Multi-domain constrained kernel estimation network for blind image super resolution

Learning-based super resolution (SR) has made remarkable progress in improving image quality compared to traditional methods. However, most of these algorithms assume ideal and known image degradation processes, such as bicubic downsampling. As a result, the performance of these algorithms significantly decreases when the degradation kernel changes in low-resolution images. Therefore, blind SR networks that can estimate the degradation kernel for each image are more adaptable in realistic scenarios. Hence, how to estimate accurate degradation kernels efficiently plays an extremely important role. Nevertheless, previous blind SR designs only use image information to constrain the kernel estimation network and train the network during the inference process, resulting in limited performance and very slow runtime. In this paper, we are the first to impose constraints for the kernel estimation network in both the image domain and kernel domain to effectively optimize estimated degradation kernels. Furthermore, an efficient multi-stage network structure is leveraged to accelerate inference speed while producing high-quality kernels. The results of evaluation experiments on publicly available datasets and realistic scenarios show that, the SR network based on the proposed design can not only produce state-of-the-art high-resolution SR images but also achieve a runtime of 0.03 s for kernel estimation when a low-resolution image is enlarged by 4 times on one NVIDIA 2080Ti GPU platform.