High-resolution Patches Research Articles

Nuclear norm regularized coding with local position-patch and nonlocal similarity for face hallucination

Learning-based face hallucination methods have received much attention and progress in past few decades. Specially, position-patch based approaches have been proposed to replace the probabilistic graph-based or manifold learning-based ones. As opposed to the existing patch based methods, where the input image patch matrix is converted into vectors before combination coefficients calculation, in this paper, we propose to directly use the image matrix based regression model for combination coefficients computation to preserve the essential structural information of the input patch matrix. For each input low-resolution (LR) patch matrix, its combination coefficients over the training image patch matrices at the same position can be computed. Then the corresponding high-resolution (HR) patch matrix can be obtained with the LR training patches replaced by the corresponding HR ones. The nonlocal self-similarities are finally utilized to further improve the hallucination performance. Various experimental results on standard face databases indicate that our proposed method outperforms some state-of-the-art algorithms in terms of both visual quantity and objective metrics.

Texture Reconstruction Guided by a High-Resolution Patch.

In this paper, we aim at super-resolving a low-resolution texture under the assumption that a high-resolution patch of the texture is available. To do so, we propose a variational method that combines two approaches that are texture synthesis and image reconstruction. The resulting objective function holds a nonconvex energy that involves a quadratic distance to the low-resolution image, a histogram-based distance to the high-resolution patch, and a nonlocal regularization that links the missing pixels with the patch pixels. As for the histogram-based measure, we use a sum of Wasserstein distances between the histograms of some linear transformations of the textures. The resulting optimization problem is efficiently solved with a primal-dual proximal method. Experiments show that our method leads to a significant improvement, both visually and numerically, with respect to the state-of-the-art algorithms for solving similar problems.

Single Image Super-Resolution Using Global Regression Based on Multiple Local Linear Mappings.

Super-resolution (SR) has become more vital, because of its capability to generate high-quality ultra-high definition (UHD) high-resolution (HR) images from low-resolution (LR) input images. Conventional SR methods entail high computational complexity, which makes them difficult to be implemented for up-scaling of full-high-definition input images into UHD-resolution images. Nevertheless, our previous super-interpolation (SI) method showed a good compromise between Peak-Signal-to-Noise Ratio (PSNR) performances and computational complexity. However, since SI only utilizes simple linear mappings, it may fail to precisely reconstruct HR patches with complex texture. In this paper, we present a novel SR method, which inherits the large-to-small patch conversion scheme from SI but uses global regression based on local linear mappings (GLM). Thus, our new SR method is called GLM-SI. In GLM-SI, each LR input patch is divided into 25 overlapped subpatches. Next, based on the local properties of these subpatches, 25 different local linear mappings are applied to the current LR input patch to generate 25 HR patch candidates, which are then regressed into one final HR patch using a global regressor. The local linear mappings are learned cluster-wise in our off-line training phase. The main contribution of this paper is as follows: Previously, linear-mapping-based conventional SR methods, including SI only used one simple yet coarse linear mapping to each patch to reconstruct its HR version. On the contrary, for each LR input patch, our GLM-SI is the first to apply a combination of multiple local linear mappings, where each local linear mapping is found according to local properties of the current LR patch. Therefore, it can better approximate nonlinear LR-to-HR mappings for HR patches with complex texture. Experiment results show that the proposed GLM-SI method outperforms most of the state-of-the-art methods, and shows comparable PSNR performance with much lower computational complexity when compared with a super-resolution method based on convolutional neural nets (SRCNN15). Compared with the previous SI method that is limited with a scale factor of 2, GLM-SI shows superior performance with average 0.79 dB higher in PSNR, and can be used for scale factors of 3 or higher.

A New Single Image Super-Resolution Method Based on the Infinite Mixture Model

As a powerful nonparametric Bayesian model, the infinite mixture model has been successfully used in machine learning and computer vision. The success of the infinite mixture model owes to the capability clustering and density estimation. In this paper, we propose a nonparametric Bayesian model for single-image super-resolution. Specifically, we combine the Dirichlet process and Gaussian process regression for estimating the distribution of the training patches and modeling the relationship between the low-resolution and high-resolution patches: 1) the proposed method groups the training patches by utilizing the clustering property of Dirichlet process; 2) the proposed method relates the low-resolution and high-resolution patches by predicting the property of Gaussian process; and 3) the mentioned two points are not independent but jointly learned. Hence, the proposed method can make full use of the nonparametric Bayesian model. First, the Dirichlet process mixture model is used to obtain more accurate clusters for training patches. Second, Gaussian process regression is established on each cluster, and this directly reduces the computational complexity. Finally, the two procedures are learned simultaneously to gain the suitable clusters with the ability of prediction. The parameters can be inferred simply via the Gibbs sampling technique. Thorough super-resolution experiments on various images demonstrate that the proposed method is superior to some state-of-the-art methods.

Super-Resolution Image Reconstruction with Improved Sparse Representation

In this paper, we present a new approach to reconstruct a high resolution (HR) image from a low resolution (LR) input image based on a two dimensional (2D) sparse method. The new method consists of three phases. Firstly, the nonlinear feature of the input LR image is divided into the linear subspace, and then LR-HR dictionaries are learned to reduce the blurred artifacts of the image. Secondly, 2D sparse representation and self-similarity are developed to strengthen and enhance the image structure. Finally, the final HR image is achieved by reconstruction of all HR patches. Simulation results demonstrated that our proposed method achieved superior results on real images, and shows various improvements in terms of PSNR and SSIM values as compared with some other competent methods.

Single Image Super-Resolution Reconstruction Technique based on A Single Hybrid Dictionary

A new sparse domain approach is proposed in this paper to realize the single image super-resolution (SR) reconstruction based upon one single hybrid dictionary, which is deduced from the mixture of both the high resolution (HR) image patch samples and the low resolution (LR) ones. Moreover, a linear model is proposed to characterize the relationship between the sparse representations of both the HR image patches and the corresponding LR ones over the same hybrid dictionary. It is shown that, the requirement on the identical sparse representation of both HR and LR image patches over the corresponding HR dictionary and the LR dictionary can be relaxed. It is unveiled that, the use of one single hybrid dictionary can not only provide a more flexible framework to keep the similar sparse characteristics between the HR patches and the corresponding degenerated LR patches, but also to accommodate their differences. On this basis, the sparse domain based SR reconstruction problem is reformulated. Moreover, the proposed linear model between the sparse representations of both the HR patch and the corresponding LR patch over the same hybrid dictionary offers us a new method to interpret the image degeneration characteristics in sparse domain. Finally, practical experimental results are presented to test and verify the proposed SR approach.

Modified sparse representation based image super-resolution reconstruction method

To improve the geometric structure and texture features of reconstructed images, a novel image super-resolution reconstruction (ISR) method based on modified sparse representation, here denoted by MSR_ISR, is discussed in this paper. In this algorithm, edge and texture features of images are synchronously considered, and the over-complete sparse dictionaries of high resolution (HR) and low resolution (LR) image patches, behaving clearer structure features, are learned by feature classification based fast sparse coding (FSC) algorithm. A LR image is first preprocessed by contourlet transform method to denoise unknown noise. Furthermore, four gradient feature images of the LR image preprocessed are extracted. For HR image patches, the edge features are extracted by Canny operator. Then using these edge pixel values as the benchmark to determine whether each image patch’s center value is equal to one of edge pixel values, then the edge and texture image patches can be marked out. For gradient image patches, they are first classified by the extreme learning machine (ELM) classifier, thus, corresponding to the class label sequence of LR image patches, the HR image features can also be classified. Furthermore, using FSC algorithm based on the k-means singular value decomposition (K-SVD) model, the edge and texture feature classification dictionaries of HR and LR image patches can be trained. Utilized HR and LR dictionaries trained, a LR image can be reconstructed well. In test, the artificial LR images, namely degraded natural images, are used to testify our ISR method proposed. Utilized the signal noise ratio (SNR) criterion to estimate the quality of reconstructed images and compared with other algorithms of the common K-SVD, FSC and FSC based K-SVD without considering feature classification technique, simulation results show that our method has clear improvement in visual effect and can retain well image edge and texture features.

Single image super-resolution by directionally structured coupled dictionary learning

In this paper, a new algorithm is proposed based on coupled dictionary learning with mapping function for the problem of single-image super-resolution. Dictionaries are designed for a set of clustered data. Data is classified into directional clusters by correlation criterion. The training data is structured into nine clusters based on correlation between the data patches and already developed directional templates. The invariance of the sparse representations is assumed for the task of super-resolution. For each cluster, a pair of high-resolution and low-resolution dictionaries are designed along with their mapping functions. This coupled dictionary learning with a mapping function helps in strengthening the invariance of sparse representation coefficients for different resolution levels. During the reconstruction phase, for a given low-resolution patch a set of directional clustered dictionaries are used, and the cluster is selected which gives the least sparse representation error. Then, a pair of dictionaries with mapping functions of that cluster are used for the high-resolution patch approximation. The proposed algorithm is compared with earlier work including the currently top-ranked super-resolution algorithm. By the proposed mechanism, the recovery of directional fine features becomes prominent.

Facial Image Hallucination Through Coupled-Layer Neighbor Embedding

As the facial image captured by a low-cost camera is typically very low resolution (LR), blurring, and noisy, traditional neighbor-embedding-based facial image hallucination methods from one single manifold (i.e., the LR image manifold) fail to reliably estimate the intention geometrical structure, consequently leading to a bias to the image reconstruction result. In this paper, we introduce the notion of neighbor embedding (NE) from the LR and the high-resolution (HR) image manifolds simultaneously and propose a novel NE model, termed the coupled-layer NE (CLNE), for facial image hallucination. CLNE differs substantially from other NE models in that it has two layers: the LR and the HR layers. The LR layer in this model is the local geometrical structure of the LR patch manifold, which is characterized by the reconstruction weights of the LR patches; the HR layer is the intrinsic geometry that can geometrically constrain the reconstruction weights. With this coupled-constraint paradigm between the adaptation of the LR layer and the HR one, CLNE can achieve a more robust NE through iteratively updating the LR patch reconstruction weights and the estimated HR patch. The experimental results in simulation and real conditions confirm that the proposed method outperforms the related state-of-the-art methods in both quantitative and visual comparisons.

Fast Learning-Based Single Image Super-Resolution

We present a learning-based single image super-resolution (SISR) method to obtain a high resolution (HR) image from a single given low resolution (LR) image. Our method gives more accurate results while also testing (runs) and training faster with a smaller number of training samples compared to other methods. We posed SISR as a problem of estimating a function to predict the pixels of an HR patch using its corresponding LR pixels and their spatial neighborhood. We studied the impact of varying the input LR and output HR patch sizes and gained the following insights: reconstruction accuracy for a given output HR patch size improves when input LR patch size is increased, but the improvement saturates after including a few extra layers of LR pixels. Moreover, HR reconstruction accuracy is the highest when the output HR patch is restricted to only that which corresponds to one LR pixel. We used zero component analysis as a pre-processing step to enhance the estimation optimization energy on perceptually salient features such as edges. We tapped into the ability of polynomial neural networks to hierarchically learn refinements of a function that maps LR to HR patches. Accurate HR reconstruction with small input and output patch sizes not only makes learning more efficient, it also indicates that SISR is a highly local problem. In contrast, a recently proposed and related technique using convolutional neural networks needs much larger training set and longer training time because of larger input-output patch sizes and a computationally expensive learning algorithm.

Example-based super-resolution for enhancing spatial resolution of medical images.

This paper proposes an effective exampled-based super-resolution (SR) method to improve the spatial resolution of medical image heavily corrupted by noise. Based on the sparsity of patches, the reconstruction of a high-resolution (HR) patch from each low-resolution (LR) input patch can be performed with the help of a database, by solving a non-negative sparse optimization problem. The challenge is to effectively solve this problem in case of a large size database. To cope with this issue, we propose a metric to measure the similarity between image patches based on the Earth Mover's Distance (EMD) in order to select only the most similar candidates that will be used in the optimization problem. Experimental results demonstrate the efficiency of our algorithm over many existing SR methods, especially in case of noise-corrupted LR image.

Text-image super-resolution through anchored neighborhood regression with multiple class-specific dictionaries

In the dictionary-based image super-resolution (SR) methods, the resolution of the input image is enhanced using a dictionary of low-resolution (LR) and high-resolution (HR) image patches. Typically, a single dictionary is learned from all the patches in the training set. Then, the input LR patch is super-resolved using its nearest LR patches and their corresponding HR patches in the dictionary. In this paper, we propose a text-image SR method using multiple class-specific dictionaries. Each dictionary is learned from the patches of images of a specific character in the training set. The input LR image is segmented into text lines and characters, and the characters are preliminarily classified. Likewise, overlapping patches are extracted from the input LR image. Then, each patch is super-resolved through the anchored neighborhood regression, using n class-specific dictionaries corresponding to the top-n classification results of the character containing the patch. The final HR image is generated by aggregating all the super-resolved patches. Our method achieves significant improvements in visual image quality and OCR accuracy, compared to the related dictionary-based SR methods. This confirms the effectiveness of applying the preliminary character classification results and multiple class-specific dictionaries in text-image SR.

Super-Resolution of Dynamic Scenes Using Sampling Rate Diversity.

In earlier work, we proposed a super-resolution (SR) method that required the availability of two low resolution (LR) sequences corresponding to two different sampling rates, where images from one sequence were used as a basis to represent the polyphase components (PPCs) of the high resolution (HR) image, while the other LR sequences provided the reference LR image (to be super-resolved). The (simple) algorithm implemented by Salem and Yagle is only applicable when the scene is static. In this paper, we recast our approach to SR as a two-stage example-based algorithm to process dynamic scenes. We employ feature selection to create, from the LR frames, local LR dictionaries to represent PPCs of HR patches. To enforce sparsity, we implement Gaussian generative models as an efficient alternative to L1-norm minimization. Estimation errors are further reduced using what we refer to as the anchors, which are based on the relationship between PPCs corresponding to different sampling rates. In the second stage, we revert to simple single frame SR (applied to each frame), using HR dictionaries extracted from the super-resolved sequence of the previous stage. The second stage is thus a reiteration of the sparsity coding scheme, using only one LR sequence, and without involving PPCs. The ability of the modified algorithm to super-resolve challenging LR sequences reintroduces sampling rate diversity as a prerequisite of robust multiframe SR.

A new framework for remote sensing image super-resolution: Sparse representation-based method by processing dictionaries with multi-type features

Jointly representing an image with different types of features is proposed in feature extraction stage.Multi-type feature dictionaries are obtained in sparse representation stage to capture the different structures of the image.Multiple HR patches can be estimated with multi-type feature dictionaries from one LR patch.A strategy is proposed to integrate those estimated HR patches by adaptively adjusting the weights. Remote sensing images play an important role in many practical applications, however, due to the physical limitations of remote sensing devices, it is difficult to obtain images at an expecting high resolution level. Acquiring high-resolution(HR) images from the original low-resolution(LR) ones with super-resolution(SR) methods has always been an attractive proposition in embedded systems including various kinds of tablet PC and smart phone. SR methods based on sparse representation have been successfully used in processing remote sensing images, however, they have two major problems in common. First, they use only one type of image features to represent the low resolution(LR) images. However, one single type of features cannot accurately represent an image due to the diverse structures of the image, as a result, artifacts would be produced simultaneously. Second, many dictionary learning methods try to build a universal dictionary with only one single type of features. However, apparently, a dictionary with a single type of features is not enough to capture the different structures of a remote sensing image, without any doubt, the resultant image would turn out to be a poor one. To overcome the problems above, we propose a new framework for remote sensing image super resolution: sparse representation-based SR method by processing dictionaries with multi-type features. First, in order to represent the remote sensing image more accurately, different types of features are extracted from images. Second, to achieve a better performance, various dictionaries with multi-type features are learned to capture the essential structures of the image. Then, it's proposed to adaptively control the weights of the high resolution(HR) patches obtained by different dictionaries. Numerous experiments validate that this proposed framework brings better results in terms of both objective quantitation and visual perception than other compared algorithms.

Consistent Coding Scheme for Single-Image Super-Resolution Via Independent Dictionaries

In this paper, we present a unified frame based on collaborative representation (CR) for single-image super-resolution (SR), which learns low-resolution (LR) and high-resolution (HR) dictionaries independently in the training stage and adopts a consistent coding scheme (CCS) to guarantee the prediction accuracy of HR coding coefficients during SR reconstruction. The independent LR and HR dictionaries are learned based on CR with $l_{2}$ -norm regularization, which can well describe the corresponding LR and HR patch space, respectively. Furthermore, a mapping function is learned to map LR coding coefficients onto the corresponding HR coding coefficients. Propagation filtering can achieve smoothing over an image while preserving image context like edges or textural regions. Moreover, to preserve the edge structures of a super-resolved image and suppress artifacts, a propagation filtering-based constraint and image nonlocal self-similarity regularization are introduced into the SR reconstruction framework. Experimental comparison with state-of-the-art single image SR algorithms validates the effectiveness of proposed approach.

Learning-based compressed sensing for infrared image super resolution

This paper presents an infrared image super-resolution method based on compressed sensing (CS). First, the reconstruction model under the CS framework is established and a Toeplitz matrix is selected as the sensing matrix. Compared with traditional learning-based methods, the proposed method uses a set of sub-dictionaries instead of two coupled dictionaries to recover high resolution (HR) images. And Toeplitz sensing matrix allows the proposed method time-efficient. Second, all training samples are divided into several feature spaces by using the proposed adaptive k-means classification method, which is more accurate than the standard k-means method. On the basis of this approach, a complex nonlinear mapping from the HR space to low resolution (LR) space can be converted into several compact linear mappings. Finally, the relationships between HR and LR image patches can be obtained by multi-sub-dictionaries and HR infrared images are reconstructed by the input LR images and multi-sub-dictionaries. The experimental results show that the proposed method is quantitatively and qualitatively more effective than other state-of-the-art methods.

Single Image Super-Resolution Using a Joint GMM Method.

Single image super-resolution (SR) algorithms based on joint dictionaries and sparse representations of image patches have received significant attention in the literature and deliver the state-of-the-art results. Recently, Gaussian mixture models (GMMs) have emerged as favored prior for natural image patches in various image restoration problems. In this paper, we approach the single image SR problem by using a joint GMM learnt from concatenated vectors of high and low resolution patches sampled from a large database of pairs of high resolution and the corresponding low resolution images. Covariance matrices of the learnt Gaussian models capture the inherent correlations between high and low resolution patches, which are utilized for inferring high resolution patches from given low resolution patches. The proposed joint GMM method can be interpreted as the GMM analogue of joint dictionary-based algorithms for single image SR. We study the performance of the proposed joint GMM method by comparing with various competing algorithms for single image SR. Our experiments on various natural images demonstrate the competitive performance obtained by the proposed method at low computational cost.

Single-Image Super-Resolution by Subdictionary Coding and Kernel Regression

In this paper, we present a new learning-based single-image super-resolution (SR) approach, inspired by existing sparse representation-based methods. As a promising image modeling theory, sparse representation has been effectively applied to solve the image SR problem, usually with the use of pretrained coupled or semi-coupled dictionaries. In our proposed method, we train independent dictionaries for high-resolution (HR) and low-resolution (LR) image patches to endow them more flexibility of expression. We use local subdictionaries to adaptively code image patches, which can characterize image local structures better and ensure the sparsity property of the image. Furthermore, we use kernel regression to relate HR and LR coding coefficients to capture and map the intrinsic nonlinear relationship between them. Such mapping is of central importance in the image SR problem, because high-order statistics play a significant role in the reconstruction of the detail structure of an HR image. The proposed model is generic for image SR in terms of two categories of blurring kernel. Experimental results show that our method can effectively reconstruct image details and outperform state-of-the-art algorithms in both quantitative and visual comparisons.

Scaling up of Low Resolution Images using Super Resolution Techniques & Performing Intensity Correction for Medical Imaging

The diagnosis of illness and certain underlying conditions can be effectively done by the use of Medical imaging. Higher resolution images are used to increase the diagnostic capabilities of the medical practitioner which leads to early and effective diagnosis of an ailment. Although a lot of advanced devices like Computerized Tomography (CT), Magnetic Resonance Imaging (MRI) etc. are currently available, the problem of Noise, Blur limits the overall ability of these devices to produce higher resolution images. A solution which can be proposed is the use of Super Resolution (SR) techniques which can be used for processing of such images. In this paper we make use of intensity correction of input medical images. The super resolution methods are done patch wise. The input images are divided into patches and dictionaries containing high and low resolution patches are obtained. Using the k-SVD algorithm for dictionary learning and OMP method for image super resolution reconstruction, the final high resolution image is obtained. The combination of intensity correction and super resolution leads to computationally and visually better results.

Fast multisensor infrared image super-resolution scheme with multiple regression models

We propose a fast super-resolution scheme with multiple regression models.This study combines the information from VI images and IR images.This study divides training patches into multiple clusters by K-means clustering.We propose a soft-assignment based multiple regression method. High resolution (HR) infrared (IR) images play an important role in many areas. However, it is difficult to obtain images at a desired resolution level because of the limitation of hardware and image environment. Therefore, improving the spatial resolution of infrared images has become more and more urgent. Methods based on sparse coding have been successfully used in single-image super-resolution (SR) reconstruction. However, the existing sparse representation-based SR method for infrared (IR) images usually encounter three problems. First, IR images always lack detailed information, which leads to unsatisfying IR image reconstruction results with conventional method. Second, the existing dictionary learning methods in SR aim at learning a universal and over-complete dictionary to represent various image structures. A large number of different structural patterns exist in an image, whereas one dictionary is not capable of capturing all of the different structures. Finally, the optimization for dictionary learning and image reconstruction requires a highly intensive computation, which restricts the practical application in real-time systems. To overcome these problems, we propose a fast IR image SR scheme. Firstly, we integrate the information from visible (VI) images and IR images to improve the resolution of IR images because images acquired by different sensors provide complementary information for the same scene. Second, we divide the training patches into several clusters, then the multiple dictionaries are learned for each cluster in order to provide each patch with a more accurate dictionary. Finally, we propose an method of Soft-assignment based Multiple Regression (SMR). SMR reconstructs the high resolution (HR) patch by the dictionaries corresponding to its K nearest training patch clusters. The method has a low level of computational complexity and may be readily suitable for real-time processing applications. Numerous experiments validate that this scheme brings better results in terms of quantization and visual perception than many state-of-the-art methods, while at the same time maintains a relatively low level of time complexity. Since the main computation of this scheme is matrix multiplication, it will be easily implemented in FPGA system.