Single Image Super-resolution Task Research Articles

Learning Image Profile Enhancement and Denoising Statistics Priors for Single-Image Super-Resolution

Single-image super-resolution (SR) has been widely used in computer vision applications. The reconstruction-based SR methods are mainly based on certain prior terms to regularize the SR problem. However, it is very challenging to further improve the SR performance by the conventional design of explicit prior terms. Because of the powerful learning ability, deep convolutional neural networks (CNNs) have been widely used in single-image SR task. However, it is difficult to achieve further improvement by only designing the network architecture. In addition, most existing deep CNN-based SR methods learn a nonlinear mapping function to directly map low-resolution (LR) images to desirable high-resolution (HR) images, ignoring the observation models of input images. Inspired by the split Bregman iteration (SBI) algorithm, which is a powerful technique for solving the constrained optimization problems, the original SR problem is divided into two subproblems: 1) inversion subproblem and 2) denoising subproblem. Since the inversion subproblem can be regarded as an inversion step to reconstruct an intermediate HR image with sharper edges and finer structures, we propose to use deep CNN to capture low-level explicit image profile enhancement prior (PEP). Since the denoising subproblem aims to remove the noise in the intermediate image, we adopt a simple and effective denoising network to learn implicit image denoising statistics prior (DSP). Furthermore, the penalty parameter in SBI is adaptively tuned during the iterations for better performance. Finally, we also prove the convergence of our method. Thus, the deep CNNs are exploited to capture both implicit and explicit image statistics priors. Due to SBI, the SR observation model is also leveraged. Consequently, it bridges between two popular SR approaches: 1) learning-based method and 2) reconstruction-based method. Experimental results show that the proposed method achieves the state-of-the-art SR results.

Single image super-resolution reconstruction based on multi-scale feature mapping adversarial network

Single image super-resolution (SISR) aims to reconstruct a high-resolution image from a degraded low-resolution image. In recent years, the super-resolution methods based on convolutional neural network (CNN) have achieved promising performance on SISR task, indicating that CNN is a viable approach to image super-resolution reconstruction. The one limitation of the current SISR methods is that many methods use the pixel-wise loss. It is well known that the pixel-wise loss cannot well recover high-frequency details even if the high peak signal-to-noise ratio (PSNR) can be obtained. Some other methods purely focus on restoring more details, which resulted in poor PSNR score and high-frequency noise. In this paper, we proposed a multi-component loss function based on pixel-wise loss, perceptual loss and adversarial loss for a multi-scale feature mapping generator network for SISR image reconstruction model. We evaluated our method on commonly used benchmarks and compared it with other SISR methods. The results showed that our method could achieve the better balance between the high-frequency detail and stable spatial structure generation.

Deep Learning Based Single Image Super-resolution: A Survey

Single image super-resolution has attracted increasing attention and has a wide range of applications in satellite imaging, medical imaging, computer vision, security surveillance imaging, remote sensing, objection detection, and recognition. Recently, deep learning techniques have emerged and blossomed, producing “the state-of-the-art” in many domains. Due to their capability in feature extraction and mapping, it is very helpful to predict high-frequency details lost in low-resolution images. In this paper, we give an overview of recent advances in deep learning-based models and methods that have been applied to single image super-resolution tasks. We also summarize, compare and discuss various models from the past and present for comprehensive understanding and finally provide open problems and possible directions for future research.

Convolutional Sparse Coding Using Wavelets for Single Image Super-Resolution

In this paper, we propose the convolutional sparse coding based model in the wavelet domain for the task of single image super-resolution (SISR). The conventional sparse coding based approaches work on overlapping image patches and use the dictionary atoms to sparse code an image patch. Further, at the final stage, an overlap-add mechanism is used to get the final high-resolution image estimate. However, these algorithms fail to take into account the consistency present in the overlapping patches which limits their performance. We propose the use of wavelet integrated convolutional sparse coding approach where instead of dictionary atoms we utilize the convolution summations between the learned filters and their mappings for sparse representation based SISR. The use of wavelets is proposed owing to their unique directional and compact features. A pair of filters are learned along with a mapping function for each wavelet sub-band to exploit the consistency among patches. The proposed wavelet integrated convolutional sparse coding model helps capture useful contextual information. The proposed model is evaluated on publicly available datasets for different scale-up parameters. To show the efficacy of the proposed model we compare it with recent state-of-the-art algorithms. The visual results along with the quantitative ones indicate that the proposed model performs well for the tasks of super-resolution.

A Novel Multiconnected Convolutional Network for Super-Resolution

Convolutional neural networks exhibit superior performance for single image super-resolution (SISR) tasks. However, as the network grows deeper, features from the earlier layers are impeded or less used in later layers. In SISR, the earlier layers are mainly composed of local features that are essential to the task. In this letter, we present a novel multiconnected convolutional network for SISR tasks by enhancing the combination of both low- and high-level features. We design a structure built on multiconnected blocks to extract diversified and complicated features via the concatenation of low-level features to high-level features. In addition to stacking multiconnected blocks, a long skip-connection is implemented to further aggregate features of the first layer and a specific later layer. Furthermore, we employ a flexible two-parameter loss function to optimize the training process. The proposed method yields state-of-the-art performance both in terms of quantitative metrics and visual quality. The method also outperforms existing methods on datasets via unknown degrading operators, indicating an excellent generalization ability.

Learning Temporal Dynamics for Video Super-Resolution: A Deep Learning Approach.

Video super-resolution (SR) aims at estimating a high-resolution (HR) video sequence from a low-resolution (LR) one. Given that deep learning has been successfully applied to the task of single image SR, which demonstrates the strong capability of neural networks for modeling spatial relation within one single image, the key challenge to conduct video SR is how to efficiently and effectively exploit the temporal dependency among consecutive LR frames other than the spatial relation. However, this remains challenging because complex motion is difficult to model and can bring detrimental effects if not handled properly. We tackle the problem of learning temporal dynamics from two aspects. First, we propose a temporal adaptive neural network that can adaptively determine the optimal scale of temporal dependency. Inspired by the Inception module in GoogLeNet [1], filters of various temporal scales are applied to the input LR sequence before their responses are adaptively aggregated, in order to fully exploit the temporal relation among consecutive LR frames. Second, we decrease the complexity of motion among neighboring frames using a spatial alignment network that can be end-to-end trained with the temporal adaptive network and has the merit of increasing the robustness to complex motion and the efficiency compared to competing image alignment methods. We provide a comprehensive evaluation of the temporal adaptation and the spatial alignment modules. We show the temporal adaptive design considerably improve SR quality over its plain counterparts, and the spatial alignment network is able to attain comparable SR performance with the sophisticated optical flow based approach, but requires much less running time. Overall our proposed model with learned temporal dynamics is shown to achieve state-of-the-art SR results in terms of not only spatial consistency but also temporal coherence on public video datasets. More information can be found in.

Coupled dictionary learning in wavelet domain for Single-Image Super-Resolution

In this paper a coupled dictionary learning mechanism with mapping function is proposed in the wavelet domain for the task of Single-Image Super-Resolution. Sparsity is used as the invariant feature for achieving super-resolution. Instead of using a single dictionary multiple compact dictionaries are proposed in the wavelet domain. Such dictionaries will exhibit the properties of the wavelet transform such as compactness, directionality and redundancy. Six pairs of dictionaries are designed using a coupled dictionary mechanism with mapping function which helps in strengthening the similarity between the sparse coefficients. Low-resolution image is assumed as the approximation image of the first-level wavelet decomposition. High resolution is achieved by estimating the wavelet sub-bands of this low-resolution image by dictionary learning and sparsity. The proposed algorithm outperforms a well-known spatial domain and wavelet domain algorithm as evaluated on the existing comparative parameters such as structural similarity index measure and peak signal-to-noise ratio.

Single image super-resolution via blind blurring estimation and dictionary learning

Learning-based methods have been becoming the mainstream of single image super resolution (SR) technologies. This kind of methods makes it effective to generate a high resolution (HR) image from a single low resolution (LR) image. There exists, however, two main problems with these methods: the quality of training data and the computational demand. We propose a novel framework for single image SR tasks in this paper, which consists of blur kernel estimation (BKE) and dictionary learning. BKE is utilized for improving the quality of training samples and realized by minimizing the dissimilarity between cross-scale patches iteratively. Couple dictionaries are trained by improved training samples before sparse recovery. More important is that a selective patch processing (SPP) strategy is adopted in BKE and sparse recovery, which brings more accurate BKE results and immensely reduces time consumption of the entire process. The experiments show that the proposed method produces more precise BKE estimation and better SR recovery than several typical SR algorithms at a higher efficiency.

Single image super-resolution via blind blurring estimation and anchored space mapping

It has been widely acknowledged that learning-based super-resolution (SR) methods are effective to recover a high resolution (HR) image from a single low resolution (LR) input image. However, there exist two main challenges in learning-based SR methods currently: the quality of training samples and the demand for computation. We proposed a novel framework for single image SR tasks aiming at these issues, which consists of blind blurring kernel estimation (BKE) and SR recovery with anchored space mapping (ASM). BKE is realized via minimizing the cross-scale dissimilarity of the image iteratively, and SR recovery with ASM is performed based on iterative least square dictionary learning algorithm (ILS-DLA). BKE is capable of improving the compatibility of training samples and testing samples effectively and ASM can reduce consumed time during SR recovery radically. Moreover, a selective patch processing (SPP) strategy measured by average gradient amplitude |grad| of a patch is adopted to accelerate the BKE process. The experimental results show that our method outruns several typical blind and non-blind algorithms on equal conditions.

Superresolution from a Single Noisy Image by the Median Filter Transform

The task of single-image superresolution is to obtain a high resolution image from a low resolution input. Many strategies have achieved considerable success by learning from image databases so as to find suitable replacements for the missing information. However, little effort has been devoted to coping with significant amounts of noise in the input. Noise makes the problem even harder, since its distribution is unknown in many practical cases. Here an approach is proposed to solve this problem irrespective of the noise type, which supports both integer and fractional zoom factors. It is based on the application of the median filter on parallelogram shaped windows chosen according to a suitable probability distribution. The resulting outputs are then median filtered again to obtain the final output. Experimental results are reported for artificial and natural images under Gaussian and impulsive uniform noise. It is found that the proposed approach outperforms several state-of-the-art single-image superre...

Image Super-Resolution Via Double Sparsity Regularized Manifold Learning

Over the past few years, high resolutions have been desirable or essential, e.g., in online video systems, and therefore, much has been done to achieve an image of higher resolution from the corresponding low-resolution ones. This procedure of recovering/rebuilding is called single-image super-resolution (SR). Performance of image SR has been significantly improved via methods of sparse coding. That is to say, the image frame patch can be sparse linear combinations of basis elements. However, most of these existing methods fail to consider the local geometrical structure in the space of the training data. To take this crucial issue into account, this paper proposes a method named double sparsity regularized manifold learning (DSRML). DSRML can preserve the properties of the aforementioned local geometrical structure by employing manifold learning, e.g., locally linear embedding. Based on a large amount of experimental results, DSRML is demonstrated to be more robust and more effective than previous efforts in the task of single-image SR.