Image Super-resolution Problem Research Articles

Single image super-resolution via an iterative reproducing kernel Hilbert space method.

Image super-resolution, a process to enhance image resolution, has important applications in satellite imaging, high definition television, medical imaging, etc. Many existing approaches use multiple low-resolution images to recover one high-resolution image. In this paper, we present an iterative scheme to solve single image super-resolution problems. It recovers a high quality high-resolution image from solely one low-resolution image without using a training data set. We solve the problem from image intensity function estimation perspective and assume the image contains smooth and edge components. We model the smooth components of an image using a thin-plate reproducing kernel Hilbert space (RKHS) and the edges using approximated Heaviside functions. The proposed method is applied to image patches, aiming to reduce computation and storage. Visual and quantitative comparisons with some competitive approaches show the effectiveness of the proposed method.

Image Super-Resolution Based on Structure-Modulated Sparse Representation

Sparse representation has recently attracted enormous interests in the field of image restoration. The conventional sparsity-based methods enforce sparse coding on small image patches with certain constraints. However, they neglected the characteristics of image structures both within the same scale and across the different scales for the image sparse representation. This drawback limits the modeling capability of sparsity-based super-resolution methods, especially for the recovery of the observed low-resolution images. In this paper, we propose a joint super-resolution framework of structure-modulated sparse representations to improve the performance of sparsity-based image super-resolution. The proposed algorithm formulates the constrained optimization problem for high-resolution image recovery. The multistep magnification scheme with the ridge regression is first used to exploit the multiscale redundancy for the initial estimation of the high-resolution image. Then, the gradient histogram preservation is incorporated as a regularization term in sparse modeling of the image super-resolution problem. Finally, the numerical solution is provided to solve the super-resolution problem of model parameter estimation and sparse representation. Extensive experiments on image super-resolution are carried out to validate the generality, effectiveness, and robustness of the proposed algorithm. Experimental results demonstrate that our proposed algorithm, which can recover more fine structures and details from an input low-resolution image, outperforms the state-of-the-art methods both subjectively and objectively in most cases.

Learning ramp transformation for single image super-resolution

We propose the use of explicitly identified image structure to guide the solution of the single image super-resolution (SR) problem. We treat the image as a layout of homogeneous regions, surrounded by ramp edges of a larger contrast. Ramps are characterized by the property that any path through any ramp pixel, monotonically leading from one to the other side, has monotonically increasing (or decreasing) intensity values along it. Such a ramp profile thus captures the large contrast between the two homogeneous regions. In this paper, the SR problem is viewed primarily as one of super-resolving these ramps, since the relatively homogeneous interiors can be handled using simpler methods. Our approach involves learning how these ramps transform across resolutions, and applying the learnt transformations to the ramps of a test image. To obtain our final SR reconstruction, we use the transformed ramps as priors in a regularization framework, where the traditional backprojection constraint is used as the data term. As compared to conventional edge based SR methods, our approach provides three distinct advantages: (1) Conventional edge based SR methods are based on gradients, which use 2D filters with heuristically chosen parameters and these choices result in different gradient values. This sensitivity adversely affects learning gradient domain correspondences across different resolutions. We show that ramp profiles are more adaptive, stable and therefore reliable representations for learning edge transformations across resolutions. (2) Existing gradient based SR methods are often unable to sufficiently constrain the absolute brightness levels in the image. Our approach on the other hand, operates directly in the image intensity domain, enforcing sharpness as well as brightness consistency. (3) Unlike previous gradient based methods, we also explicitly incorporate dependency between closely spaced edges while learning ramp correspondences. This allows for better recovery of contrast across thin structures such as in high spatial frequency areas. We obtain results that are sharper and more faithful to the true image color, and show almost no ringing artifacts.

Single‐frame image super‐resolution inspired by perceptual criteria

In this study, the authors consider the problem of image super-resolution (SR) in terms of the perceptual criteria. Existing SR methods treat the traditional mean-squared error (MSE) as an irreplaceable objective function. However, MSE has been widely criticised since it is inconsistent with visual perception of human beings. The perceptual criteria, including the structural similarity (SSIM) index and feature similarity (FSIM) index, have been reported to be more effective in assessing image quality. Therefore SSIM and FSIM are included for the SR task in this study. Specifically, the authors first propose to reform principal component analysis (PCA), which is named as visual perceptual PCA (VP-PCA), by adopting SSIM as the object function. Subsequently, to accomplish the SR task, the authors cluster the training data and perform VP-PCA on each cluster to calculate the coefficients. Finally, based on the principle of FSIM, the traditional SR results and the SR results using VP-PCA are combined to form our fused results. Experimental results are provided to show the superiority of the proposed method over several state-of-the-art methods in both quantitative and visual comparisons.

Dual-sparsity regularized sparse representation for single image super-resolution

Recently, by exploring the column nonlocal similarity prior among the sparse representation coefficients, the column nonlocal similarity sparse representation models for solving the ill-posed single image super-resolution (SISR) problem are attracting more and more attention. However, these conventional models consider only the prior among nonlocal similar sparse representation coefficients, and fail to consider the prior among all entries (or rows) of the sparse representation coefficient. Hence the modeling capability may be limited. In fact, if a cluster of similar representation coefficients is rearranged into a matrix in the sparse representation coefficient space, the nonlocal similarity priors exist both among columns and rows. Using the row nonlocal similarity prior, a row nonlocal similarity regularization term with l1-norm constraint is explored. By introducing it to the conventional column nonlocal similarity sparse representation model, we present a dual-sparsity regularized sparse representation (DSRSR) model. A surrogate function based iterative shrinkage algorithm is introduced to effectively solve the proposed model. Extensive experiments on SISR demonstrate that the presented model can effectively reconstruct the edge structures and suppress the noise, achieving convincing improvement over many state-of-the-art example-based methods in terms of PSNR, SSIM and visual quality.

Very Low Resolution Face Image Super-Resolution Based on DCT

This paper introduces the Very Low Resolution (VLR) problem in the fleld of face image super-resolution reconstruction. With the extensive application of the surveillance cameras, the very low resolution problem happens in many face recognition applications, which has brought great inconvenience to users. In order to research this problem, this paper proposes the improved DCT algorithm, and our flnal experimental results have compared with the common DCT algorithm, Cubic B-Spline interpolation algorithm and Baker et al’s method. Experimental results show that the improved DCT algorithm can get better performance in the very low resolution face image super-resolution reconstruction problem.

Sparse representation with morphologic regularizations for single image super-resolution

Due to the fact that natural images are inherently sparse in some domains, sparse representation has led to interesting results in image acquiring, representing, and compressing high-dimensional signals. Based on the experiences and learned priors in sparse domain from low and high resolution images, the typical ill-posed inverse problem of image super-resolution is effectively solved by the l1-norm optimization techniques. However, how to reasonably combine the sparse representation theory and the feature of natural images is still a critical issue for performances improvements of image super-resolution algorithms. Considering the fact that the different morphologic features in natural images should be regularized by different constrains in sparse domain, in this paper we present a novel sparse representation algorithm with reasonable morphologic regularization for single image super-resolution. Extensive experimental results on various natural images validate the superiority of the proposed algorithm in terms of qualitative and quantitative performance.

Single Image Super Resolution Through Neighbor Embedding Based on Field of Experts

A parametric framework, called Field of Experts (FoE), has been proposed for learning generic, expressive image priors that capture the statistics of natural scenes and can be used for a variety of machine vision tasks. In this paper we apply Field of Experts for the first time to the problem of single natural image super resolution through neighbor embedding, where we toke the coefficients of the linear filters learned by FoE as the feature vectors. Experimental results show that the proposed method not only achieve a better recovery of a single low resolution image comparing with state-of-the-art feature vectors, but also reduce the computational complexity.

Understanding Compressive Sensing and Sparse Representation-Based Super-Resolution

Recently, compressive sensing (CS) has emerged as a powerful tool for solving a class of inverse/underdetermined problems in computer vision and image processing. In this paper, we investigate the application of CS paradigms on single image super-resolution (SR) problems that are considered to be the most challenging in this class. In light of recent promising results, we propose novel tools for analyzing sparse representation-based inverse problems using redundant dictionary basis. Further, we provide novel results establishing tighter correspondence between SR and CS. As such, we gain insights into questions concerning regularizing the solution to the underdetermined problem, such as follows. 1) Is sparsity prior alone sufficient? 2) What is a good dictionary? 3) What is the practical implication of noncompliance with theoretical CS hypothesis? Unlike in other underdetermined problems that assume random down-projections, the low-resolution image formation model employed in CS-based SR is a deterministic down-projection that may not necessarily satisfy some critical assumptions of CS. We further investigate the impact of such projections in concern to the above questions.

Preconditioning for Edge-Preserving Image Super Resolution

We propose a simple preconditioning method for accelerating the solution of edge-preserving image super-resolution (SR) problems in which a linear shift-invariant point spread function is employed. Our technique involves reordering the high-resolution (HR) pixels in a similar manner to what is done in preconditioning methods for quadratic SR formulations. However, due to the edge preserving requirements, the Hessian matrix of the cost function varies during the minimization process. We develop an efficient update scheme for the preconditioner in order to cope with this situation. Unlike some other acceleration strategies that round the displacement values between the low-resolution (LR) images on the HR grid, the proposed method does not sacrifice the optimality of the observation model. In addition, we describe a technique for preconditioning SR problems involving rational magnification factors. The use of such factors is motivated in part by the fact that, under certain circumstances, optimal SR zooms are nonintegers. We show that, by reordering the pixels of the LR images, the structure of the problem to solve is modified in such a way that preconditioners based on circulant operators can be used.

Example-based image super-resolution with class-specific predictors

Example-based super-resolution is a promising approach to solving the image super-resolution problem. However, the learning process can be slow and prediction can be inaccurate. In this paper, we present a novel learning-based algorithm for image super-resolution to improve the computational speed and prediction accuracy. Our new method classifies image patches into several classes, for each class, a class-specific predictor is designed. A class-specific predictor takes a low-resolution image patch as input and predicts a corresponding high-resolution patch as output. The performances of the class-specific predictors are evaluated using different datasets formed by face images and natural-scene images. We present experimental results which demonstrate that the new method provides improved performances over existing methods.

Exact Image Super-Resolution for Pure Translational Motion and Shift-Invariant Blur

In this work, a special case of image super-resolution problem where the only type of motion is global translational motion and the blurs are shift-invariant is investigated. The necessary conditions for exact reconstruction of the original image by using finite impulse-response reconstruction filters are developed. If the number of low-resolution images is larger than a threshold and the blur functions meet a certain property, then a reconstruction filter set for exact super-resolution can be generated. If the conditions are not fully satisfied, then the high-resolution image is reconstructed with some error.

NEW TECHNIQUES TO CONQUER THE IMAGE RESOLUTION ENHANCEMENT PROBLEM

This paper presents some new techniques for high resolution (HR) image processing and compares between them. The paper focuses on two main topics, image interpolation and image super- resolution. By image interpolation, we mean extracting an HR image from a single Degraded low resolution (LR) image. Polynomial based image interpolation is reviewed. Some new techniques for adaptive image interpolation and inverse image interpolation are presented. The other topic treated in this paper is image super-resolution. By image super resolution, we mean extracting a single HR image either from multiple observations or multiple frames. The paper focuses on the problem of image super resolution using wavelet fusion and presents several super resolution reconstruction algorithms based on the idea of wavelet fusion.

Image superresolution using fractal coding

We address the problem of image superresolution and present a novel approach to single-frame superresolution using fractal image coding. The proposed approach takes great advantage of the properties of fractals—resolution independence, similarity preservation, and nonlinear operation—which are suited for image superresolution, specifically for image restoration and magnification. The idea of our work is to estimate the fractal code of the original image from its degraded blurred and noised observation and decode it at a higher resolution, and all the strategies are performed in the fractal image coding frame- work. To achieve this, we employ an adaptive fractal coding scheme in the frequency domain, and further, we introduce an overlapping partition scheme to remove the blocky artifacts and improve the reconstruction quality. Experiments on simulated and real images show that the result- ing fractal-based superresolution method yields superior performance to conventional single-frame superresolution methods. © 2008 Society of Photo-

Super Resolution of Images and Video

This book focuses on the super resolution of images and video. The authors’ use of the term super resolution (SR) is used to describe the process of obtaining a high resolution (HR) image, or a sequence of HR images, from a set of low resolution (LR) observations. This process has also been referred to in the literature as resolution enhancement (RE). SR has been applied primarily to spatial and temporal RE, but also to hyperspectral image enhancement. This book concentrates on motion based spatial RE, although the authors also describe motion free and hyperspectral image SR problems. Also examined is the very recent research area of SR for compression, which consists of the intentional downsampling, during pre-processing, of a video sequence to be compressed and the application of SR techniques, during post-processing, on the compressed sequence.