Dictionary Pair Learning Research Articles

Removal of Optically Thick Clouds From High-Resolution Satellite Imagery Using Dictionary Group Learning and Interdictionary Nonlocal Joint Sparse Coding

In this paper, we propose a method for cloud removal in a cloud-contaminated high-resolution (HR) optical satellite image with two kinds of auxiliary images of different types: a low-resolution (LR) optical satellite composite image and a synthetic aperture radar (SAR) image. In the proposed method, we assume that cloud-contaminated and cloud-free regions have been detected accurately, then dictionary group learning (DGL) is used to establish structure correspondences between HR, LR, and SAR data from cloud-free patches, while interdictionary nonlocal joint sparse coding (INJSC) is used to estimate the universal representation coefficients of patches contaminated by clouds, and finally, cloud-contaminated HR patches can be reconstructed with their universal coefficients and the HR dictionary learned from DGL process. In this way, the missing information in the cloud-contaminated HR image can be reconstructed patch by patch. The proposed method is tested on a series of experiments on both simulated and real data. Experimental results show that both DGL and INJSC are beneficial to better reconstructing the missing information. This method is also compared against our previous work on the same topic, which adopted dictionary pair learning (DPL) and sparse coding (SC) to recover the missing information and achieved state-of-the-art performance at that time. The comparison shows that the method proposed in this paper significantly outperforms the previous one.

Multi-Kernel Low-Rank Dictionary Pair Learning for Multiple Features Based Image Classification

Dictionary learning (DL) is an effective feature learning technique, and has led to interesting results in many classification tasks. Recently, by combining DL with multiple kernel learning (which is a crucial and effective technique for combining different feature representation information), a few multi-kernel DL methods have been presented to solve the multiple feature representations based classification problem. However, how to improve the representation capability and discriminability of multi-kernel dictionary has not been well studied. In this paper, we propose a novel multi-kernel DL approach, named multi-kernel low-rank dictionary pair learning (MKLDPL). Specifically, MKLDPL jointly learns a kernel synthesis dictionary and a kernel analysis dictionary by exploiting the class label information. The learned synthesis and analysis dictionaries work together to implement the coding and reconstruction of samples in the kernel space. To enhance the discriminability of the learned multi-kernel dictionaries, MKLDPL imposes the low-rank regularization on the analysis dictionary, which can make samples from the same class have similar representations. We apply MKLDPL for multiple features based image classification task. Experimental results demonstrate the effectiveness of the proposed approach.

Learning Heterogeneous Dictionary Pair with Feature Projection Matrix for Pedestrian Video Retrieval via Single Query Image

Person re-identification (re-id) plays an important role in video surveillance and forensics applications. In many cases, person re-id needs to be conducted between image and video clip, e.g., re-identifying a suspect from large quantities of pedestrian videos given a single image of him. We call re-id in this scenario as image to video person re-id (IVPR). In practice, image and video are usually represented with different features, and there usually exist large variations between frames within each video. These factors make matching between image and video become a very challenging task. In this paper, we propose a joint feature projection matrix and heterogeneous dictionary pair learning (PHDL) approach for IVPR. Specifically, PHDL jointly learns an intra-video projection matrix and a pair of heterogeneous image and video dictionaries. With the learned projection matrix, the influence of variations within each video to the matching can be reduced. With the learned dictionary pair, the heterogeneous image and video features can be transformed into coding coefficients with the same dimension, such that the matching can be conducted using coding coefficients. Furthermore, to ensure that the obtained coding coefficients have favorable discriminability, PHDL designs a point-to-set coefficient discriminant term. Experiments on the public iLIDS-VID and PRID 2011 datasets demonstrate the effectiveness of the proposed approach.

Accurate classification of brain gliomas by discriminate dictionary learning based on projective dictionary pair learning of proton magnetic resonance spectra.

Proton magnetic resonance spectroscopy is a powerful noninvasive technique that complements the structural images of cMRI, which aids biomedical and clinical researches, by identifying and visualizing the compositions of various metabolites within the tissues of interest. However, accurate classification of proton magnetic resonance spectroscopy is still a challenging issue in clinics due to low signal-to-noise ratio, overlapping peaks of metabolites, and the presence of background macromolecules. This paper evaluates the performance of a discriminate dictionary learning classifiers based on projective dictionary pair learning method for brain gliomas proton magnetic resonance spectroscopy spectra classification task, and the result were compared with the sub-dictionary learning methods. The proton magnetic resonance spectroscopy data contain a total of 150 spectra (74 healthy, 23 grade II, 23 grade III, and 30 grade IV) from two databases. The datasets from both databases were first coupled together, followed by column normalization. The Kennard-Stone algorithm was used to split the datasets into its training and test sets. Performance comparison based on the overall accuracy, sensitivity, specificity, and precision was conducted. Based on the overall accuracy of our classification scheme, the dictionary pair learning method was found to outperform the sub-dictionary learning methods 97.78% compared with 68.89%, respectively. Copyright © 2016 John Wiley & Sons, Ltd.

Projective dictionary pair learning for EEG signal classification in brain computer interface applications

Electroencephalogram (EEG) based brain-computer interface (BCI) is a useful communication tool between human brain and external devices. Accurate and effective EEG classification plays an important role in performance of BCI applications. In this paper, we propose a dictionary pair learning (DPL) method for EEG signal classification. In this method, we can learn a dictionary without costly L0 and L1 calculation and sparse coefficients have been calculated by linear projection instead of nonlinear sparse coding. We analyzed the performance of new method using EEG data from IIIa and IVa databases of BCI competition III. Experimental results showed that proposed method provides higher classification performance compared with other dictionary learning methods such as label consistent K Singular value decomposition (LC-KSVD). Based on our results, accuracy rates are as follows: 81.25%, 100%, 60.2%, 83.04% and 79.37% for subjects “aa”, “al”, “av”, “aw” and “ay”, respectively from IVa database. Also, the average accuracy rate of 85.7% has been achieved for two-class classification of IIIa database.

A Novel Dictionary Learning Model with PT-HLBP for Palmprint Recognition

A novel projective dictionary pair learning (PDPL) model with statistical local features for palmprint recognition is proposed. Pooling technique is used to enhance the invariance of hierarchical local binary pattern (PT-HLBP) for palmprint feature extraction. PDPL is employed to learn an analysis dictionary and a synthesis dictionary which are utilized for image discrimination and representation. The proposed algorithm has been tested by the Hong Kong Polytechnic University (PolyU) database (v2) and ideal recognition accuracy can be achieved. Experimental results indicate that the algorithm not only greatly reduces the time complexity in training and testing phase, but also exhibits good robustness for image rotation and corrosion.

Dictionary Pair Learning on Grassmann Manifolds for Image Denoising.

Image denoising is a fundamental problem in computer vision and image processing that holds considerable practical importance for real-world applications. The traditional patch-based and sparse coding-driven image denoising methods convert 2D image patches into 1D vectors for further processing. Thus, these methods inevitably break down the inherent 2D geometric structure of natural images. To overcome this limitation pertaining to the previous image denoising methods, we propose a 2D image denoising model, namely, the dictionary pair learning (DPL) model, and we design a corresponding algorithm called the DPL on the Grassmann-manifold (DPLG) algorithm. The DPLG algorithm first learns an initial dictionary pair (i.e., the left and right dictionaries) by employing a subspace partition technique on the Grassmann manifold, wherein the refined dictionary pair is obtained through a sub-dictionary pair merging. The DPLG obtains a sparse representation by encoding each image patch only with the selected sub-dictionary pair. The non-zero elements of the sparse representation are further smoothed by the graph Laplacian operator to remove the noise. Consequently, the DPLG algorithm not only preserves the inherent 2D geometric structure of natural images but also performs manifold smoothing in the 2D sparse coding space. We demonstrate that the DPLG algorithm also improves the structural SIMilarity values of the perceptual visual quality for denoised images using the experimental evaluations on the benchmark images and Berkeley segmentation data sets. Moreover, the DPLG also produces the competitive peak signal-to-noise ratio values from popular image denoising algorithms.

Evaluation of local spatial–temporal features for cross-view action recognition

Local spatial–temporal feature-based representation is extremely popular for human action recognition. Many spatial–temporal salient point detectors and descriptors have been proposed. Although the promising results have been achieved for action recognition recently, there still exist two severe problems: (1) there is lack of systematic evaluation of local spatial–temporal features on cross-view action recognition; (2) there is lack of a baseline method especially for the task of cross-view action recognition, which can adaptively bridge different feature spaces from multiple views for this cross-domain task. In this paper, we evaluate four popular spatial–temporal features (STIP, Cuboids, MoSIFT, HoG3D) with the framework of transferable dictionary pair learning. This framework can first learn one transferable dictionary pair in both unsupervised and supervised settings. Then, training samples in the source view and testing samples in the target view can be represented with corresponding source and target dictionary respectively to get sparse feature representations, which is used to training classifier for action recognition. In this way, it can map the features from different views into the same feature space to handle the cross-domain task. The evaluation of four spatial–temporal features and the framework of transferable dictionary pair learning are implemented on the popular multi-view human action dataset, IXMAS. The comparative experiments against the representative methods further demonstrate the superiority of this framework on cross-view human action recognition.

Improving the Spatial Resolution of Landsat TM/ETM+ Through Fusion With SPOT5 Images via Learning-Based Super-Resolution

To take advantage of the wide swath width of Landsat Thematic Mapper (TM)/Enhanced Thematic Mapper Plus (ETM+) images and the high spatial resolution of Système Pour l'Observation de la Terre 5 (SPOT5) images, we present a learning-based super-resolution method to fuse these two data types. The fused images are expected to be characterized by the swath width of TM/ETM+ images and the spatial resolution of SPOT5 images. To this end, we first model the imaging process from a SPOT image to a TM/ETM+ image at their corresponding bands, by building an image degradation model via blurring and downsampling operations. With this degradation model, we can generate a simulated Landsat image from each SPOT5 image, thereby avoiding the requirement for geometric coregistration for the two input images. Then, band by band, image fusion can be implemented in two stages: 1) learning a dictionary pair representing the high- and low-resolution details from the given SPOT5 and the simulated TM/ETM+ images; 2) super-resolving the input Landsat images based on the dictionary pair and a sparse coding algorithm. It is noteworthy that the proposed method can also deal with the conventional spatial and spectral fusion of TM/ETM+ and SPOT5 images by using the learned dictionary pairs. To examine the performance of the proposed method of fusing the swath width of TM/ETM+ and the spatial resolution of SPOT5, we illustrate the fusion results on the actual TM images and compare with several classic pansharpening methods by assuming that the corresponding SPOT5 panchromatic image exists. Furthermore, we implement the classification experiments on both actual images and fusion results to demonstrate the benefits of the proposed method for further classification applications.

Local learned dictionaries optimized to edge orientation for inverse halftoning.

A method is proposed for fully restoring local image structures of an unknown continuous-tone patch from an input halftoned patch with homogenously distributed dot patterns, based on a locally learned dictionary pair via feature clustering. First, many training sets consisting of paired halftone and continuous-tone patches are collected, and then histogram-of- oriented-gradient (HOG) feature vectors that describe the edge orientations are calculated from every continuous-tone patch, to group the training sets. Next, a dictionary learning algorithm is separately conducted on the categorized training sets, to obtain the halftone and continuous-tone dictionary pairs, optimized to edge-oriented patch representation. Finally, an adaptively smoothing filter is applied to the input halftone patch, to predict the HOG feature vector of an unknown continuous-tone patch, and to select one of the previously learned dictionary pairs, based on the Euclidean distance between the HOG mean feature vectors of the grouped training sets and the predicted HOG vector. In addition to using the local dictionary pairs, a patch fusion technique is used to reduce some artifacts, such as color noise and overemphasized edges on smooth regions. Experimental results show that the use of the paired dictionary selected by the local edge orientation and patch fusion technique not only reduced the artifacts in smooth regions, but also provided well expressed fine details and outlines, especially in the areas of textures, lines, and regular patterns.

Super-resolution restoration of MMW image based on sparse representation method

This paper presents a new super-resolution (SR) restoration method of a single millimeter wave image (MMW), based on sparse single representation. A single MMW image is in fact a low-resolution one that is viewed as a downsampled version of a high-resolution image, whose image patches are assumed to be well represented as a sparse linear combination of elements from an appropriately chosen overcomplete dictionary. The theoretical results from compressed sensing ensure that under mild conditions, the sparse representation can be correctly restored from the downsampled signal. Inspired by this idea, a sparse representation for each image path of the MMW image with hardly low-resolution is sought, and then the coefficients of this sparse representation are used to generate the high-resolution output. The high- and low-resolution dictionary pair can be obtained by utilizing the modified fast sparse coding (FSC) algorithm, and it is a more compact representation of the image patch pairs, which hardly reduces the computational cost. In tests, image patches of several high-resolution natural images with different classes and their corresponding low-resolution versions are firstly used to train the dictionary pair. Further, utilizing the learned dictionary pair, the MMW image is restored efficiently. For the restored natural images, the image quality is measured by single noise ratio (SNR), while for the MMW image, it is not suitable to use the SNR criterion because much noise existed in the MMW image. So, the restored MMW image’s quality is measured by the relative SNR (RSNR). Compared with other SR restoration methods of bicubic, Lucy–Richardson (L–R) and neighbor embedding (NE), the simulation experimental results testify that the sparse representation is indeed effective in the super-resolution restoration task of the MMW image.

Spatio-spectral fusion of satellite images based on dictionary-pair learning

This paper proposes a novel spatial and spectral fusion method for satellite multispectral and hyperspectral (or high-spectral) images based on dictionary-pair learning. By combining the spectral information from sensors with low spatial resolution but high spectral resolution (LSHS) and the spatial information from sensors with high spatial resolution but low spectral resolution (HSLS), this method aims to generate fused data with both high spatial and spectral resolution. Based on the sparse non-negative matrix factorization technique, this method first extracts spectral bases of LSHS and HSLS images by making full use of the rich spectral information in LSHS data. The spectral bases of these two categories data then formulate a dictionary-pair due to their correspondence in representing each pixel spectra of LSHS data and HSLS data, respectively. Subsequently, the LSHS image is spatial unmixed by representing the HSLS image with respect to the corresponding learned dictionary to derive its representation coefficients. Combining the spectral bases of LSHS data and the representation coefficients of HSLS data, fused data are finally derived which are characterized by the spectral resolution of LSHS data and the spatial resolution of HSLS data. The experiments are carried out by comparing the proposed method with two representative methods on both simulation data and actual satellite images, including the fusion of Landsat/ETM+ and Aqua/MODIS data and the fusion of EO-1/Hyperion and SPOT5/HRG multispectral images. By visually comparing the fusion results and quantitatively evaluating them in term of several measurement indices, it can be concluded that the proposed method is effective in preserving both the spectral information and spatial details and performs better than the comparison approaches.

Image Super-Resolution Via Sparse Representation

This paper presents a new approach to single-image super-resolution, based on sparse signal representation. Research on image statistics suggests that image patches can be well-represented as a sparse linear combination of elements from an appropriately chosen over-complete dictionary. Inspired by this observation, we seek a sparse representation for each patch of the low-resolution input, and then use the coefficients of this representation to generate the high-resolution output. Theoretical results from compressed sensing suggest that under mild conditions, the sparse representation can be correctly recovered from the downsampled signals. By jointly training two dictionaries for the low- and high-resolution image patches, we can enforce the similarity of sparse representations between the low resolution and high resolution image patch pair with respect to their own dictionaries. Therefore, the sparse representation of a low resolution image patch can be applied with the high resolution image patch dictionary to generate a high resolution image patch. The learned dictionary pair is a more compact representation of the patch pairs, compared to previous approaches, which simply sample a large amount of image patch pairs, reducing the computational cost substantially. The effectiveness of such a sparsity prior is demonstrated for both general image super-resolution and the special case of face hallucination. In both cases, our algorithm generates high-resolution images that are competitive or even superior in quality to images produced by other similar SR methods. In addition, the local sparse modeling of our approach is naturally robust to noise, and therefore the proposed algorithm can handle super-resolution with noisy inputs in a more unified framework.