Coupled Dictionary Learning Research Articles

Multimodal Image Super-Resolution via Joint Sparse Representations Induced by Coupled Dictionaries

Real-world data processing problems often involve various image modalities associated with a certain scene, including RGB images, infrared images or multi-spectral images. The fact that different image modalities often share certain attributes, such as certain edges, textures and other structure primitives, represents an opportunity to enhance various image processing tasks. This paper proposes a new approach to construct a high-resolution (HR) version of a low-resolution (LR) image given another HR image modality as reference, based on joint sparse representations induced by coupled dictionaries. Our approach, which captures the similarities and disparities between different image modalities in a learned sparse feature domain in \emph{lieu} of the original image domain, consists of two phases. The coupled dictionary learning phase is used to learn a set of dictionaries that couple different image modalities in the sparse feature domain given a set of training data. In turn, the coupled super-resolution phase leverages such coupled dictionaries to construct a HR version of the LR target image given another related image modality. One of the merits of our sparsity-driven approach relates to the fact that it overcomes drawbacks such as the texture copying artifacts commonly resulting from inconsistency between the guidance and target images. Experiments on real multimodal images demonstrate that incorporating appropriate guidance information via joint sparse representation induced by coupled dictionary learning brings notable benefits in the super-resolution task with respect to the state-of-the-art. Of particular relevance, the proposed approach also demonstrates better robustness than competing deep-learning-based methods in the presence of noise.

Spectral Super Resolution of Hyperspectral Images via Coupled Dictionary Learning

High-spectral resolution imaging systems play a critical role in the identification and characterization of objects in a scene of interest. Unfortunately, multiple factors impair spectral resolution, as in the case of modern snapshot spectral imagers that associate each hyperpixel with a specific spectral band. In this paper, we introduce a novel postacquisition computational technique aiming to enhance the spectral dimensionality of imaging systems by exploiting the mathematical frameworks of sparse representations and dictionary learning. We propose a coupled dictionary learning model which considers joint feature spaces, composed of low- and high-spectral resolution hypercubes, in order to achieve spectral superresolution performance. We formulate our spectral coupled dictionary learning optimization problem within the context of the alternating direction method of multipliers, and we manage to update the involved quantities via closed-form expressions. In addition, we consider a realistic spectral subsampling scenario, taking into account the spectral response functions of different satellites. Moreover, we apply our spectral superresolution algorithm on real satellite data acquired by Landsat-8 and Sentinel-2 sensors. Finally, we have investigated the problem of hyperspectral image unmixing using the recovered high-spectral resolution data cube, and we are able to demonstrate that the proposed scheme provides significant value in hyperspectral image understanding techniques. Experimental results demonstrate the ability of the proposed approach to synthesize high-spectral-resolution 3-D hypercubes, achieving better performance compared to state-of-the-art resolution enhancement methods.

Low-Dose Computed Tomography Image Super-Resolution Reconstruction via Random Forests.

Aiming at reducing computed tomography (CT) scan radiation while ensuring CT image quality, a new low-dose CT super-resolution reconstruction method based on combining a random forest with coupled dictionary learning is proposed. The random forest classifier finds the optimal solution of the mapping relationship between low-dose CT (LDCT) images and high-dose CT (HDCT) images and then completes CT image reconstruction by coupled dictionary learning. An iterative method is developed to improve robustness, the important coefficients for the tree structure are discussed and the optimal solutions are reported. The proposed method is further compared with a traditional interpolation method. The results show that the proposed algorithm can obtain a higher peak signal-to-noise ratio (PSNR) and structural similarity index measurement (SSIM) and has better ability to reduce noise and artifacts. This method can be applied to many different medical imaging fields in the future and the addition of computer multithreaded computing can reduce time consumption.

Multi-view coupled dictionary learning for person re-identification

In recent years, person re-identification is becoming an important technique, which can be applied in computer vision, pedestrian tracking and intelligent monitoring. Due to the large variations of visual appearance caused by view angle, pose changing, light changing, background clutter and occlusion, person re-identification is very challenging. In practice, there exist large differences among different types of features and among different cameras. To improve the favorable representation of different features, we propose a multi-view based coupled dictionary pair learning framework, which can learn dictionary pairs for multiple categories of features, e.g., the color features, texture features and hybrid features etc. Specifically, with the learned color feature dictionary pair, we can obtain the color feature representation coefficients of each person from different cameras. The texture feature dictionary pair seeks to learn the texture feature representation coefficients of each person from both cameras. The hybrid feature dictionary pair aims to learn the hybrid feature coefficients for each person. The learned coupled dictionary pairs can demonstrate the intrinsic relationship of different cameras and different types of features. When the resolution of image is too low, the texture information will be lost to some extent. There is few high-resolution person dataset so far, we contribute a newly collected dataset, named High-Resolution Pedestrian re-Identification Dataset (HRPID) on the campus of Wuhan University. The size of person images is normalized to 230 × 560 pixels, which is bigger than existing person re-identification datasets. Experimental results on a new dataset and two public pedestrian datasets demonstrate that our proposed approach can perform better than the other competing methods.

ISAR HRRP Super Resolution Reconstruction Based on Couple Dictionary Learning

It is important for Inverse Synthetic Aperture Radar (ISAR) to improve its resolution, which can get more critical information. In this paper, an ISAR super-resolution imaging algorithm based on joint dictionary learning is proposed, which means we find two special sets of sparse signals from numerous high and low resolution ISAR images respectively, this two sets of sparse elements which are also called dictionaries can represent all the original signal linearly. Through the learned dictionaries, we can achieve the super resolution of ISAR images. A coupled dictionary learning algorithm based on Restricted Boltzmann Machine (RBM) is designed to learn high and lowresolution dictionaries. Finally, low-resolution ISAR images are sparsely reconstructed based on dictionaries. The suggested approach has lower computational complexity and higher anti-noise performance.

Sparsity-based inverse halftoning via semi-coupled multi-dictionary learning and structural clustering

Inverse halftoning is the restoration of a continuous-tone image from its halftone version, which is a critical process for halftone transform, digital archive management and high precision identification of halftone. In this paper, a novel inverse halftoning method based on semi-coupled multi-dictionary learning is proposed to address the cross-style image restoration from halftone images to continuous-tone images. By using semi-coupled multi-dictionary learning, multiple dictionary pairs and their corresponding mapping functions between continuous-tone image and its halftone version could be simultaneously learned. The learned multiple dictionary pairs can well represent the structure characteristics of halftone images and continuous-tone images, respectively. In addition, the mapping functions learned by semi-coupled manner can bridge the gap between the two different style images of halftone image and continuous-tone image. Unlike the existed methods, the proposed method could effectively relax the assumption of the same sparse coding coefficients in coupled dictionary learning. To obtain more accurate mapping functions, a structural clustering method for cross-style image patches is proposed by using SUSAN (smallest univalue segment assimilating nucleus) filtering and HOG (histogram of oriented gradient) features, which can capture the similar structure features from halftone images and continuous-tone images, and thus improve the classification accurate rate of halftone image patches. The experimental results demonstrate that the proposed method can restore higher quality continuous-tone images than that produced by the state-of-the-art methods, which not only reduce the screen noise in smooth regions, but also provide well fine details and clear edges.

Monocular 3D facial shape reconstruction from a single 2D image with coupled-dictionary learning and sparse coding

Monocular 3D face reconstruction from a single image has been an active research topic due to its wide applications. It has been demonstrated that the 3D face can be reconstructed efficiently using a PCA-based subspace model for facial shape representation and facial landmarks for model parameter estimation. However, due to the limited expressiveness of the subspace model and the inaccuracy of landmark detection, most existing methods are not robust to pose and illumination variation. To overcome this limitation, this work proposes a coupled-dictionary model for parametric facial shape representation and a two-stage framework for 3D face reconstruction from a single 2D image by using facial landmarks. Motivated by image super-resolution, the proposed coupled-model consists of two dictionaries for the sparse and the dense 3D facial shapes, respectively. In the first stage, the sparse 3D face is estimated from facial landmarks by using partial least-squares regression. In the second stage, the dense 3D face is reconstructed by 3D super-resolution on the estimated sparse 3D face. Comprehensive experimental evaluations demonstrate that the proposed coupled-dictionary model outperforms the PCA-based subspace model in 3D face modeling accuracy and that the proposed framework achieves much lower reconstruction error on facial images with pose and illumination variations compared to state-of-the-art algorithms. Moreover, qualitative analysis demonstrates that the proposed method is generalizable to different types of data, including facial images, portraits, and facial sketches.

Recovery of Soil Moisture Active Passive (SMAP) Instrument's Active Measurements via Coupled Dictionary Learning

Recovery of Soil Moisture Active Passive (SMAP) Instrument's Active Measurements via Coupled Dictionary Learning

SLR: Semi-Coupled Locality Constrained Representation for Very Low Resolution Face Recognition and Super Resolution

Although face recognition algorithms have been greatly successful recently, in real applications of very low-resolution (VLR) images, both super-resolution (SR) and recognition tasks are more challenging than those in high-resolution (HR) images. Given the rare discriminative information in VLR images, the one-to-many mapping relationship between HR and VLR images degrades the SR and recognition performances. In this paper, we propose a novel semi-coupled dictionary learning scheme to promote discriminative and representative abilities for face recognition and SR simultaneously by relaxing coupled dictionary learning. Specifically, we use semi-coupled locality-constrained representation to enhance the consistency between VLR and HR local manifold geometries, thereby overcoming the negative effects of one-to-many mapping. Given the learned task-oriented mapping function, we feed these discriminative features into a collaborative representation-based classifier to output their labels, and combine a locality-induced approach to hallucinate the HR images. Extensive experimental results demonstrate that the proposed approach outperforms a number of state-of-the-art face recognition and SR algorithms.

Image super-resolution via two stage coupled dictionary learning

Example-based image super-resolution aims to establish a learning model for generating the high resolution image from the coupled training pairs, which is an active study area of mobile media in the recent modern communication. In this paper, we proposed a novel example-based method to address the single image super-resolution problem, where the training pairs are selected from a large amount of natural images. The main idea of our method is to reconstruct the high resolution by a two stage-based scheme. In the first stage, one dictionary is learned to represent the coarse high resolution image from its low version, and the other is trained within the same coding as the coarse image to recover texture details. And then, to further enhance the fine edges in image, a similar dictionary learning scenario are done about the synthesis high resolution image and its fine structure in residual component. Extensive experimental results on some benchmark test images show the advantage of our method compare with other excellent ones.

Coupled dictionary learning in wavelet domain for Single-Image Super-Resolution

In this paper a coupled dictionary learning mechanism with mapping function is proposed in the wavelet domain for the task of Single-Image Super-Resolution. Sparsity is used as the invariant feature for achieving super-resolution. Instead of using a single dictionary multiple compact dictionaries are proposed in the wavelet domain. Such dictionaries will exhibit the properties of the wavelet transform such as compactness, directionality and redundancy. Six pairs of dictionaries are designed using a coupled dictionary mechanism with mapping function which helps in strengthening the similarity between the sparse coefficients. Low-resolution image is assumed as the approximation image of the first-level wavelet decomposition. High resolution is achieved by estimating the wavelet sub-bands of this low-resolution image by dictionary learning and sparsity. The proposed algorithm outperforms a well-known spatial domain and wavelet domain algorithm as evaluated on the existing comparative parameters such as structural similarity index measure and peak signal-to-noise ratio.

Single image super-resolution using coupled dictionary learning and cross domain mapping

In this paper, a new algorithm for single image super resolution using coupled wavelet and spatial domain dictionary pairs is proposed. The standard deviation parameter, which is approximately scale invariant for low and high resolution patch pairs is employed for clustering. A pair of online coupled dictionaries is learned for each cluster using a low resolution image. The standard deviation measure of a low resolution patch is used to select the appropriate cluster dictionary pair for reconstructing the high resolution counterpart. Experimental results show that the performance of the proposed algorithm is superior to the existing methods in terms of objective and subjective quality measures. The objective image quality is measured in terms of PSNR and SSIM. This paper also proposed an extended algorithm, based on selective sparse representation over a set of coupled dictionary pair. The extended algorithm applies the coupled dictionary based sparse framework for patches having high standard deviation. Whereas, low complexity patch collaging method is used to super resolve low standard deviation valued patches. It is found empirically that a large percentage of patches have low standard deviation values. Moreover, the selective approach significantly reduces the computational complexity without losing the overall reconstruction quality.

Coupled Dictionary Learning for Target Recognition in SAR Images

In this letter, we propose a novel classification strategy called the coupled dictionary learning for target recognition in synthetic aperture radar (SAR) images. First, we train structured synthesis dictionaries to reflect the difference among each category. Second, we introduce a shared dictionary to reduce the effect of common features, such as the high similarity caused by specular reflection. Finally, we use the analysis dictionary to improve the efficiency of recognition by eliminating the constraint of the $l_{0}$ -norm or $l_{1}$ -norm of sparse code. Experimental results on Moving and Stationary Target Acquisition and Recognition data set indicate that our method can achieve better performance in SAR target recognition than the state-of-the-art methods, such as tritask joint sparse representation and CKLR. Especially, this method can be more robust when the depressions have obvious changes.

Robust coupled dictionary learning with ℓ1-norm coefficients transition constraint for noisy image super-resolution

Conventional coupled dictionary learning approaches are designed for noiseless image super-resolution (SR), but quite sensitive to noisy images. We find that the cause is the commonly used ℓF-norm coefficients transition term. In this paper, we propose a robust ℓ1-norm solution by introducing two sub-terms: LR coefficient sparsity constraint term and HR coefficient conversion term, which are able to prevent the noise transmission from noisy input to output. By incorporating our simple yet effective non-linear model inspired by auto-encoder, the proposed ℓ1-norm dictionary learning achieves a more accurate coefficients conversion. Moreover, to make the coefficients conversion more reliable in the iterative process, we bring the non-local self-similarity constraint to regularize the HR sparse coefficients updates. The improved sparse representation further enhances SR inference on both synthesized noisy and noiseless images. Using standard metrics, we show that results are significantly clearer than state-of-the-arts on noisy images and sharper on denoised images. In addition, experiments on real-world data further demonstrate the superiority of our method in practice.

Coupled Dictionary Learning for Change Detection From Multisource Data

With the increase of multisource data available from remote sensing platforms, it is demanding to develop unsupervised techniques for change detection from multisource data. The difference in imaging mechanism makes it difficult to carry out a direct comparison between multisource data in original observation spaces. Different sensors provide different descriptions on the same truth in low-dimension observation spaces, but the same truth indicates the comparability of multisource data in some high-dimensional feature spaces. Inspired by this, we try to solve this problem by transforming multisource data into a common high-dimension feature space. In this paper, an iterative coupled dictionary learning (CDL) model is proposed for multisource image change detection. This model aims to establish a pair of coupled dictionaries, one of which is responsible for the data from one sensor, whereas the other is responsible for the data from another sensor. The atoms from these two coupled dictionaries have a one-to-one correspondence at the same location. Such a property guarantees the transferability of the reconstruction coefficients between bitemporal patch pairs and provides us a desired mechanism to bridge multisource data and highlight changes. The contributions can be summarized as follows: CDL is designed to explore the intrinsic difference of multisource data for change detection in a high-dimension feature space, and an iterative scheme for unsupervised sample selection is proposed to keep the purity of training samples and gradually optimize the current coupled dictionaries. The experimental results have demonstrated the feasibility, effectiveness, and robustness of the proposed framework.

Robust low-dose dynamic cerebral perfusion CT image restoration via coupled dictionary learning scheme.

Dynamic cerebral perfusion x-ray computed tomography (PCT) imaging has been advocated to quantitatively and qualitatively assess hemodynamic parameters in the diagnosis of acute stroke or chronic cerebrovascular diseases. However, the associated radiation dose is a significant concern to patients due to its dynamic scan protocol. To address this issue, in this paper we propose an image restoration method by utilizing coupled dictionary learning (CDL) scheme to yield clinically acceptable PCT images with low-dose data acquisition. Specifically, in the present CDL scheme, the 2D background information from the average of the baseline time frames of low-dose unenhanced CT images and the 3D enhancement information from normal-dose sequential cerebral PCT images are exploited to train the dictionary atoms respectively. After getting the two trained dictionaries, we couple them to represent the desired PCT images as spatio-temporal prior in objective function construction. Finally, the low-dose dynamic cerebral PCT images are restored by using a general DL image processing. To get a robust solution, the objective function is solved by using a modified dictionary learning based image restoration algorithm. The experimental results on clinical data show that the present method can yield more accurate kinetic enhanced details and diagnostic hemodynamic parameter maps than the state-of-the-art methods.

Single image super-resolution by directionally structured coupled dictionary learning

In this paper, a new algorithm is proposed based on coupled dictionary learning with mapping function for the problem of single-image super-resolution. Dictionaries are designed for a set of clustered data. Data is classified into directional clusters by correlation criterion. The training data is structured into nine clusters based on correlation between the data patches and already developed directional templates. The invariance of the sparse representations is assumed for the task of super-resolution. For each cluster, a pair of high-resolution and low-resolution dictionaries are designed along with their mapping functions. This coupled dictionary learning with a mapping function helps in strengthening the invariance of sparse representation coefficients for different resolution levels. During the reconstruction phase, for a given low-resolution patch a set of directional clustered dictionaries are used, and the cluster is selected which gives the least sparse representation error. Then, a pair of dictionaries with mapping functions of that cluster are used for the high-resolution patch approximation. The proposed algorithm is compared with earlier work including the currently top-ranked super-resolution algorithm. By the proposed mechanism, the recovery of directional fine features becomes prominent.

Enhancing inverse halftoning via coupled dictionary training

Inverse halftoning is a challenging problem in image processing. Traditionally, this operation is known to introduce visible distortions into reconstructed images. This paper presents a learning-based method that performs a quality enhancement procedure on images reconstructed using inverse halftoning algorithms. The proposed method is implemented using a coupled dictionary learning algorithm, which is based on a patchwise sparse representation. Specifically, the training is performed using image pairs composed by images restored using an inverse halftoning algorithm and their corresponding originals. The learning model, which is based on a sparse representation of these images, is used to construct two dictionaries. One of these dictionaries represents the original images and the other dictionary represents the distorted images. Using these dictionaries, the method generates images with a smaller number of distortions than what is produced by regular inverse halftone algorithms. Experimental results show that images generated by the proposed method have a high quality, with less chromatic aberrations, blur, and white noise distortions.

Alzheimer's disease detection via automatic 3D caudate nucleus segmentation using coupled dictionary learning with level set formulation

Background and objectiveThis paper presents a novel method for Alzheimer's disease classification via an automatic 3D caudate nucleus segmentation. MethodsThe proposed method consists of segmentation and classification steps. In the segmentation step, we propose a novel level set cost function. The proposed cost function is constrained by a sparse representation of local image features using a dictionary learning method. We present coupled dictionaries: a feature dictionary of a grayscale brain image and a label dictionary of a caudate nucleus label image. Using online dictionary learning, the coupled dictionaries are learned from the training data. The learned coupled dictionaries are embedded into a level set function. In the classification step, a region-based feature dictionary is built. The region-based feature dictionary is learned from shape features of the caudate nucleus in the training data. The classification is based on the measure of the similarity between the sparse representation of region-based shape features of the segmented caudate in the test image and the region-based feature dictionary. ResultsThe experimental results demonstrate the superiority of our method over the state-of-the-art methods by achieving a high segmentation (91.5%) and classification (92.5%) accuracy. ConclusionsIn this paper, we find that the study of the caudate nucleus atrophy gives an advantage over the study of whole brain structure atrophy to detect Alzheimer's disease.

No embedding: A novel image cryptosystem for meaningful encryption

In this paper, we propose a novel image cryptosystem, which enables to encrypt the secret images with a smaller-size cover image. Compared with the existing meaningful encryption methods, our cryptosystem has three advantages: (1) non-embedding encryption, i.e., there isn’t any data embedding into the cover image during the encryption process. (2) Our cryptosystem can simultaneously encrypt multiple secret images with one cover image, which greatly improves the security of secret images. (3) Our cryptosystem can accomplish not only the meaningful encryption, but also the meaningless encryption. Thus, people don’t switch encryption methods when meeting different encryption requirements. Our scheme leverages the popular coupled dictionary learning and compressive sensing techniques to accomplish the whole task. Specifically, we use the coupled dictionaries to build connection between the cover image and the secret image, and apply the compressive sensing to decrypt the secret image. To demonstrate the effectiveness of the proposed cryptosystem, a series of experiments are conducted. Experimental results on gray images and colorful RGB images verify its superiority.