Multi-frame Image Super-resolution Research Articles

Low-Rank Total Variation for Multi-frame Image Super-resolution

The methods based on regularization are the mainstream of super resolution research due to the characteristics of natural images. Even most of them are based on the total variation (TV) method, the characteristics of natural images cannot be expressed completely by this way. According to low rank property of natural image statistics, a new algorithm is proposed to restore more high-frequency details by combining both of them. Compare to other methods, experimental results show that the proposed algorithm can achieve a better reconstruction quality.

A new denoising model for multi-frame super-resolution image reconstruction

Multi-frame image super-resolution (SR) aims to combine the sub-pixel information from a sequence of low-resolution (LR) images to build a high-resolution (HR) one. SR techniques usually suffers from annoying restoration artifacts such as noise, jagged edges, and staircasing effect. In this paper, we aim to increase the performance of SR reconstitution under a variational framework using adaptive diffusion-based regularization term. We propose a new tensor based diffusion regularization that takes the benefit from the diffusion model of Perona–Malik in the flat regions and use a nonlinear tensor derived from the diffusion process of Weickert filter near boundaries. Thus, the proposed SR approach can preserve important image features (sharp edges and corners) much better while avoiding artifacts. The synthetic and real experimental results show the effectiveness of the proposed regularisation compared to other methods in both quantitatively and visually.

Investigation of multi-frame image super-resolution based on adaptive self-learning methods

ABSTRACTThe goal of this paper is to introduce and demonstrate a new high-performance super-resolution (SR) method for multi-frame images. By combining learning-based and reconstruction-based SR methods, this paper proposes a multi-frame image super-resolution method based on adaptive self-learning. Using the adaptive self-learning method and recovery of high-frequency edge information, an initial high-resolution (HR) image containing effective texture information is obtained. The edge smoothness prior is then used to satisfy the global reconstruction constraint and enhance the quality of the HR image. Our results indicate that this method achieves better performance than several other methods for both simulated data and real-scene images.

A multi-frame adaptive super-resolution method using double channel and regional pixel information

Multi-frame image super-resolution (SR) aims to utilize information from a set of low resolution (LR) images to compose a high-resolution (HR) one. As it is desirable or essential in many real applications, recent years have witnessed the growing interest in the problem of multi-frame SR reconstruction. In this paper, we propose a novel multi-frame image super-resolution algorithm built on the regularization framework. The objective functional to be minimized for the regularization framework consists of a fidelity term and a regularization term. A new fidelity term is formed by combining L1 norm and L2 norm according to defined Residual Weight Parameters (RWP) and Channel Weight Parameters (CWP). A new regularization term which adopts Regional Adaptive Weight Coefficients (RAWC) is proposed to keep edge and flat regions, which are implicitly described in LR images, sharp and smooth, respectively. Thorough experimental results show the new algorithm using both fidelity and regularization terms for SR reconstruction is effective than other methods.

High performance super-resolution reconstruction of multi-frame degraded images with local weighted anisotropy and successive regularization

Multi-frame image super-resolution (SR) is a procedure which takes several observed degraded low-resolution (LR) images, and processes them together to synthesize one or more high-quality high-resolution (HR) images. Due to an insufficient number of LR images and ill-conditioned blur operators, SR image reconstruction is a typical ill-posed inverse problem. However, the regularization approach can be used as an effective means to solve this kind of problems. In this paper, we propose an algorithm for multi-frame SR reconstruction with two new types of regularization terms, termed as local weighted anisotropy regularization and successive regularization toward iteration process, which are characterized by strong ability of suppressing noise and preserving edge information in HR image reconstruction. Furthermore, an iterative refinement procedure obtained superior result by employing Bregman iteration. Experiment results show that the effectiveness of the proposed regularization method and corresponding algorithm for SR reconstruction.

A Nonlinear Anisotropic GVF-shock Diffusion Method to Multi-frame Image Super-resolution Reconstruction

This paper presents a new nonlinear anisotropic Gradient Vector Flow (GVF) shock diffusion method to multi-frame image Super-resolution (SR) reconstruction. Firstly, matrix-valued diffusion tensor is constructed based on anisotropic nonlinear structure tensor, making full use of its directional smooth characteristics. Then, an anisotropic GVF is proposed to diffuse along tangent and normal directions separately, combing with adaptively weighted shock filter to enhance edge sharpness. Experimental results show that proposed algorithm enhances image edges and suppresses noise effectively. Copyright © 2015 Binary Information Press.

Multi-frame image super resolution based on sparse coding

An image super-resolution method from multiple observation of low-resolution images is proposed. The method is based on sub-pixel accuracy block matching for estimating relative displacements of observed images, and sparse signal representation for estimating the corresponding high-resolution image, where correspondence between high- and low-resolution images are modeled by a certain degradation process. Relative displacements of small patches of observed low-resolution images are accurately estimated by a computationally efficient block matching method. The matching scores of the block matching are used to select a subset of low-resolution patches for reconstructing a high-resolution patch, that is, an adaptive selection of informative low-resolution images is realized. The proposed method is shown to perform comparable or superior to conventional super-resolution methods through experiments using various images.

Single and Multi frame Image super-resolution and its Performance Analysis: A Comprehensive Survey

resolution is the process of combining one or more low -resolution images to obtain a high -resolution image. It has been a very interesting topic for the research over the last few years. It is used for practical applications in more realistic problems faced in different areas, which range over satellite and aerial imaging of biomedical image processing as well as in daily routine life like different biometrics in offices and industries. In this survey article, more focus is given on basic algorithms and their classification based methodology used to implement it. Due to its vast scope of applications researchers are developing a novel suerresolution algorithm for a specific intention based on single and multiframe image resolution. The proposed comprehensive survey gives an overview of most of published works based on its performance analysis. In this survey, the basic concepts of the algorithms are explained and then their performance analyses through which each of these methods have developed are mentioned in detail. Furthermore, different issues in superresolution algorithms for single and multiframesuch as models and registration algorithms, optimization of the deployment of methods, improvement in quality of image . Keywordsand multi frame superresolution, Image registration, wavelet transform, Assessment of SR algorithms.

An adaptive super-resolution method based on regional pixel information and ringing artifacts suppression

Multi-frame image super-resolution (SR) aims to utilize information from a set of low-resolution (LR) images to compose a high-resolution (HR) one. In this paper, a novel multi-frame image super-resolution algorithm is proposed based on regional pixel information and ringing artifacts suppression. Firstly, a new regularization term which adopts Regional Adaptive Weight Coefficients (RAWC) is produced to keep edges and flat regions. After detailed analysis, an iterative process is given for image reconstruction. Then an adaptive term according to the local variance of iterative correction image is designed to evaluate the ringing artifacts. Finally, the original iteration is updated by adding the restraint term for better visual effects and lower noise of reconstructive HR image. Thorough experimental results show the proposed algorithm is effective for SR reconstruction and ringing artifacts suppression.

Kullback–Leibler Divergence Based Composite Prior Modeling for Bayesian Super-Resolution

This paper proposes to adaptively combine the known total variation model and more recent Frobenius norm regularization for multi-frame image super-resolution (SR). In contrast to existing literature, in this paper both the composite prior modeling and posterior variational optimization are achieved in the Bayesian framework by utilizing the Kullback---Leibler divergence, and hyper-parameters related to the composite prior and noise statistics are all determined automatically, resulting in a spatially adaptive SR reconstruction method. Experimental results demonstrate that the new approach can generate a super-resolved image with higher signal-to-noise ratio and better visual perception, not only image details better preserved but also staircase effects better suppressed.

A posterior mean approach for MRF-based spatially adaptive multi-frame image super-resolution

Multi-frame image super-resolution (SR) has been intensively studied in recent years, aiming at reconstructing high-resolution images from several degraded ones (e.g., shift, blurred, aliased, and noisy). In the literature, one of the most popular SR frameworks is the maximum a posteriori model, where a spatially homogeneous image prior and manually adjusted regularization parameter are commonly used for the entire high-resolution image, thus ignoring local spatially adaptive properties of natural images. In this paper, a posterior mean approach is proposed for spatially adaptive multi-frame image super-resolution. First, a flexible Laplacian prior is proposed incorporating both the gradient and Hessian information of images, not only able to better preserve image structures, e.g., edge, texture, but also to suppress staircase effects in the flat regions. In the subsequent, a fully Bayesian SR framework is formulated, wherein the variational Bayesian method is utilized to simultaneously estimate the high-resolution image and unknown hyper-parameters for the image prior and noise. The final experimental results show that the proposed approach is highly competitive against existing algorithms, producing a super-resolved image with higher peak signal-to-noise ratio and better visual perception.

Solving a two-colour problem by applying probabilistic approach to a full-colour multi-frame image super-resolution

Reconstruction based algorithms play an important role in the multi-frame super-resolution problem. A group of images of the same scene are fused together to produce an image with higher spatial resolution, or with more visible details in the high spatial frequency features. Demosaicing algorithms interpolate missing pixels in a raw image taken from one Charged Coupled Device (CCD) array, upsampling the number of the pixels present in the image. Since super-resolution (SR) and demosaicing are the two faces of the same problem it is natural to address them together. In this paper it is: (i) shown that correct modelling of the Bayer pattern in the generative process improves the super-resolution performance for colour images, and (ii) an algorithm that incorporates the two colour prior into the probabilistic model is designed. The algorithm presented in this paper focuses on the classes of images that have two dominant colours, i.e. most of the areas in the image are uniformly coloured. A convex optimization procedure for joint super-resolution and demosaicing is developed which outperforms state-of-the-art algorithms.

A robust multiframe super-resolution algorithm based on half-quadratic estimation with modified BTV regularization

Multiframe image super-resolution is a technique to reconstruct a high-resolution image by fusing a sequence of low-resolution images of the same scene. In this paper, we propose a new multiframe image super-resolution algorithm built on the regularization framework. The objective functional to be minimized for the regularization framework consists of a fidelity term and a regularization term. A new adaptive norm combining the advantages of traditional L1 and L2 norms is used in both terms. The fidelity term is then formed by an adaptive strategy depending on the accuracies of the estimated low-resolution image observation models. This strategy serves to adaptively weight low-resolution images according to their reliability and can add robustness in practical implementation of super-resolution. The proposed regularization term can preserve sharp edges well without producing visual artifacts. Our experimental results using both synthetic and real data show the performance improvement of the proposed algorithm over other methods.

An Improved PDE Based Super-Resolution Reconstruction Algorithm

In this paper, we present an improved partial differential equation (PDE) model for multi-frame image superresolution reconstruction. Our proposed PDE model is achieved by using an adaptively weighting function, which combines the Total Variation (TV) regularization with fourth-order partial differential equations (PDE) regularization. Experiments show the effectiveness and practicability of the proposed method and demonstrate its superiority to some existing SR methods, both in its visual effects and in quantitative terms.

Multi-frame Image Super-resolution Reconstruction based on Sparse Representation and POCS

Super-resolution image reconstruction algorithms produce a high-resolution image from one or a set of low-resolution images of the desired scene. In this paper, we present a novel two-stage super-resolution (SR) algorithm combined sparse signal representation with the projection onto convex sets (POCS). In the first stage, inspired by recent results in sparse signal representation, we get a high-resolution intermediate image based on learning dictionary method for each low-resolution image of an input image sequence. In the second stage, by fusing these high-resolution intermediate images, a higher resolution image is generated based on POCS method. Experiment results show the effectiveness of our method and the improved performance over other SR algorithms.

Multiframe image super-resolution adapted with local spatial information

Super-resolution image reconstruction, which has been a hot research topic in recent years, is a process to reconstruct high-resolution images from shifted, low-resolution, degraded observations. Among the available reconstruction frameworks, the maximum a posteriori (MAP) model is widely used. However, existing methods usually employ a fixed prior item and regularization parameter for the entire HR image, ignoring local spatially adaptive properties, and the large computation load caused by the solution of the large-scale ill-posed problem is another issue to be noted. In this paper, a block-based local spatially adaptive reconstruction algorithm is proposed. To reduce the large computation load and realize the local spatially adaptive process of the prior model and regularization parameter, first the target image is divided into several same-sized blocks and the structure tensor is used to analyze the local spatial properties of each block. Different property prior items and regularization parameters are then applied adaptively to different properties' blocks. Experimental results show that the proposed method achieves better performance than methods with a fixed prior item and regularization parameter.

A multi-frame image super-resolution method

Multi-frame image super-resolution (SR) aims to utilize information from a set of low-resolution (LR) images to compose a high-resolution (HR) one. As it is desirable or essential in many real applications, recent years have witnessed the growing interest in the problem of multi-frame SR reconstruction. This set of algorithms commonly utilizes a linear observation model to construct the relationship between the recorded LR images to the unknown reconstructed HR image estimates. Recently, regularization-based schemes have been demonstrated to be effective because SR reconstruction is actually an ill-posed problem. Working within this promising framework, this paper first proposes two new regularization items, termed as locally adaptive bilateral total variation and consistency of gradients, to keep edges and flat regions, which are implicitly described in LR images, sharp and smooth, respectively. Thereafter, the combination of the proposed regularization items is superior to existing regularization items because it considers both edges and flat regions while existing ones consider only edges. Thorough experimental results show the effectiveness of the new algorithm for SR reconstruction.