Joint Dictionary Research Articles

Context-aware joint dictionary learning for color image demosaicking

Most digital cameras are overlaid with color filter arrays (CFA) on their electronic sensors, and thus only one particular color value would be captured at every pixel location. When producing the output image, one needs to recover the full color image from such incomplete color samples, and this process is known as demosaicking. In this paper, we propose a novel context-constrained demosaicking algorithm via sparse-representation based joint dictionary learning. Given a single mosaicked image with incomplete color samples, we perform color and texture constrained image segmentation and learn a dictionary with different context categories. A joint sparse representation is employed on different image components for predicting the missing color information in the resulting high-resolution image. During the dictionary learning and sparse coding processes, we advocate a locality constraint in our algorithm, which allows us to locate most relevant image data and thus achieve improved demosaicking performance. Experimental results show that the proposed method outperforms several existing or state-of-the-art techniques in terms of both subjective and objective evaluations.

Joint Dictionary Learning for Multispectral Change Detection.

Change detection is one of the most important applications of remote sensing technology. It is a challenging task due to the obvious variations in the radiometric value of spectral signature and the limited capability of utilizing spectral information. In this paper, an improved sparse coding method for change detection is proposed. The intuition of the proposed method is that unchanged pixels in different images can be well reconstructed by the joint dictionary, which corresponds to knowledge of unchanged pixels, while changed pixels cannot. First, a query image pair is projected onto the joint dictionary to constitute the knowledge of unchanged pixels. Then reconstruction error is obtained to discriminate between the changed and unchanged pixels in the different images. To select the proper thresholds for determining changed regions, an automatic threshold selection strategy is presented by minimizing the reconstruction errors of the changed pixels. Adequate experiments on multispectral data have been tested, and the experimental results compared with the state-of-the-art methods prove the superiority of the proposed method. Contributions of the proposed method can be summarized as follows: 1) joint dictionary learning is proposed to explore the intrinsic information of different images for change detection. In this case, change detection can be transformed as a sparse representation problem. To the authors' knowledge, few publications utilize joint learning dictionary in change detection; 2) an automatic threshold selection strategy is presented, which minimizes the reconstruction errors of the changed pixels without the prior assumption of the spectral signature. As a result, the threshold value provided by the proposed method can adapt to different data due to the characteristic of joint dictionary learning; and 3) the proposed method makes no prior assumption of the modeling and the handling of the spectral signature, which can be adapted to different data.

Ground-roll separation of seismic data based on morphological component analysis in two-dimensional domain

Ground roll is an interference wave that severely degrades the signal-to-noise ratio of seismic data and affects its subsequent processing and interpretation. In this study, according to differences in morphological characteristics between ground roll and reflected waves, we use morphological component analysis based on two-dimensional dictionaries to separate ground roll and reflected waves. Because ground roll is characterized by low-frequency, low-velocity, and dispersion, we select two-dimensional undecimated discrete wavelet transform as a sparse representation dictionary of ground roll. Because of a strong local correlation of the reflected wave, we select two-dimensional local discrete cosine transform as the sparse representation dictionary of reflected waves. A sparse representation model of seismic data is constructed based on a two-dimensional joint dictionary then a block coordinate relaxation algorithm is used to solve the model and decompose seismic record into reflected wave part and ground roll part.The good effects for the synthetic seismic data and application of real seismic data indicate that when using the model, strong-energy ground roll is considerably suppressed and the waveform of the reflected wave is effectively protected.

Pedestrian detection from thermal images: A sparse representation based approach

Pedestrian detection, a key technology in computer vision, plays a paramount role in the applications of advanced driver assistant systems (ADASs) and autonomous vehicles. The objective of pedestrian detection is to identify and locate people in a dynamic environment so that accidents can be avoided. With significant variations introduced by illumination, occlusion, articulated pose, and complex background, pedestrian detection is a challenging task for visual perception. Different from visible images, thermal images are captured and presented with intensity maps based objects’ emissivity, and thus have an enhanced spectral range to make human beings perceptible from the cool background. In this study, a sparse representation based approach is proposed for pedestrian detection from thermal images. We first adopted the histogram of sparse code to represent image features and then detect pedestrian with the extracted features in an unimodal and a multimodal framework respectively. In the unimodal framework, two types of dictionaries, i.e. joint dictionary and individual dictionary, are built by learning from prepared training samples. In the multimodal framework, a weighted fusion scheme is proposed to further highlight the contributions from features with higher separability. To validate the proposed approach, experiments were conducted to compare with three widely used features: Haar wavelets (HWs), histogram of oriented gradients (HOG), and histogram of phase congruency (HPC) as well as two classification methods, i.e. AdaBoost and support vector machine (SVM). Experimental results on a publicly available data set demonstrate the superiority of the proposed approach.

Supervised Monaural Speech Enhancement Using Complementary Joint Sparse Representations

Sparse representation is one of the most well-known methods that are applied to monaural speech enhancement. In order to make full use of the relationships among speech, noise, and mixture in sparse representation for speech enhancement, this letter proposes a novel sparsity model that consists of a couple of joint sparse representations (JSRs). One JSR uses the mapping relationship between mixture and speech while the other uses that between mixture and noise. Both relationships are used to constrain the joint dictionary learning, which effectively solves the source confusion problem of traditional methods. Moreover, the latter JSR can be complementary to the former JSR, depending on the level of structure of the noise. Thus, we propose a Gini index based weighting parameter to take their complementary advantages. The experimental results show that the proposed method outperforms state-of-the-art methods using various objective measures.

Single Image Super-Resolution Using a Joint GMM Method.

Single image super-resolution (SR) algorithms based on joint dictionaries and sparse representations of image patches have received significant attention in the literature and deliver the state-of-the-art results. Recently, Gaussian mixture models (GMMs) have emerged as favored prior for natural image patches in various image restoration problems. In this paper, we approach the single image SR problem by using a joint GMM learnt from concatenated vectors of high and low resolution patches sampled from a large database of pairs of high resolution and the corresponding low resolution images. Covariance matrices of the learnt Gaussian models capture the inherent correlations between high and low resolution patches, which are utilized for inferring high resolution patches from given low resolution patches. The proposed joint GMM method can be interpreted as the GMM analogue of joint dictionary-based algorithms for single image SR. We study the performance of the proposed joint GMM method by comparing with various competing algorithms for single image SR. Our experiments on various natural images demonstrate the competitive performance obtained by the proposed method at low computational cost.

Shrunken Locally Linear Embedding for Passive Microwave Retrieval of Precipitation

This paper introduces a new Bayesian approach to the inverse problem of passive microwave rainfall retrieval. The proposed methodology relies on a regularization technique and makes use of two joint dictionaries of coincidental rainfall profiles and their corresponding upwelling spectral radiative fluxes. A sequential detection-estimation strategy is adopted, which basically assumes that similar rainfall intensity values and their spectral radiances live close to some sufficiently smooth manifolds with analogous local geometry. The detection step employs a nearest neighborhood classification rule, while the estimation scheme is equipped with a constrained shrinkage estimator to ensure stability of retrieval and some physical consistency. The algorithm is examined using coincidental observations of the active precipitation radar (PR) and passive microwave imager (TMI) on board the Tropical Rainfall Measuring Mission (TRMM) satellite. We present promising results of instantaneous rainfall retrieval for some tropical storms and mesoscale convective systems over ocean, land, and coastal zones. We provide evidence that the algorithm is capable of properly capturing different storm morphologies including high intensity rain-cells and trailing light rainfall, especially over land and coastal areas. The algorithm is also validated at an annual scale for calendar year 2013 versus the standard (version 7) radar (2A25) and radiometer (2A12) rainfall products of the TRMM satellite.

Discriminative structured dictionary learning with hierarchical group sparsity

Learning adaptive dictionaries for sparse coding has been the focus of latest research as it provides a promising way to maximize the efficiency of sparse representation. In particular, learning discriminative dictionaries rather than reconstructive ones has demonstrated significantly improved performance in pattern recognition. In this paper, a powerful method is proposed for discriminative dictionary learning. During the dictionary learning process, we enhance the discriminability of sparse codes by promoting hierarchical group sparsity and reducing linear prediction error on sparse codes. With the employment of joint within-class collaborative hierarchical sparsity, our method is able to learn adaptive dictionaries from labeled data for classification, which encourage coefficients to be sparse at both group level and singleton level and thus enforce the separability of sparse codes. Benefiting from joint dictionary and classifier learning, the discriminability of sparse codes is further strengthened. An efficient alternating iterative scheme is presented to solve the proposed model. We applied our method to face recognition, object recognition and scene classification. Experimental results have demonstrated the excellent performance of our method in comparison with existing discriminative dictionary learning approaches.

Super-resolution for Medical Image via Sparse Representation and Adaptive M-estimator.

The goal of super-resolution is to generate high-resolution images from low-resolution input images. In this paper, a combined method based on sparse signal representation and adaptive M-estimator is proposed for single-image super-resolution. With the sparse signal representation, the correlation between the sparse representation of high-resolution patches and that of low-resolution patches for the identical image is learned as a set of joint dictionaries and a set of high-resolution patches is obtained for high- and low-resolution patches. Then the dictionaries and high-resolution patches are used to produce the high-resolution image for a low-resolution single image. At the post-processing phase, the adaptive M-estimator, combining the advantages of traditional L1 and L2 norms, is used to give further processing for the resultant high-resolution image, to reduce the artefact by learning and reconstitution, and improve the performance. Three experimental results show the performance improvement of the proposed algorithm over other methods.

Guest Editorial: Ad Hoc Web Multimedia Analysis with Limited Supervision

With the popularity of social media applications and Web 2.0 techniques, there is an explosive growth of web multimedia data generated from user-sharing web sites such as Flickr, YouTube, and Facebook. The social characteristics and the increased scalability turn out to be a great challenge in the semantic understanding and retrieval of web multimedia. Though the users’ comments and tagging can be well exploited to provide more semantic cues for the web multimedia analysis, the annotations of these data contain a lot of noisy tags and are always weakly tagging. Thus, the supervision information available is limited due to the huge output space. Furthermore, for web multimedia analysis, the negative examples come from an infinite semantic space and we have no clue about the semantics these negative examples include. Thus, how to construct the generic model for each ad hoc multimedia analysis task (a.k.a. AdHocWeb Multimedia Analysis) is a challenging problem for the social media applications on the web. The research of ad hoc web multimedia analysis with limited supervision is an interesting and fundamental research area, which involves several fields, ranging from machine learning, multimedia retrieval, and computer vision to data mining. This issue consists of 11 papers, which are briefly discussed as follows. Visual content analysis is a fundamental problem in ad hoc multimedia analysis, which will facilitate the semantic understanding of multimedia data. In this issue, two papers investigate visual content analysis and its applications in super-resolution and image-based localization on mobile phone. The “Depth map Super-Resolution based on joint dictionary learning” (10.1007/s11042-014-2002-6) paper utilizes unsupervised dictionary learning for depth map super-resolution. This method transforms a low-resolution depth map to a high-resolution depth map. Different from previous depth map super-resolution methods, the proposed algorithm uses a joint dictionary learning method with both lowand high-resolution depth Multimed Tools Appl (2015) 74:463–465 DOI 10.1007/s11042-014-2419-y

Sparse representation with learned multiscale dictionary for image fusion

Sparse representation is a powerful tool to describe image, and achieves lots of state of the art results in various image processing areas. The choice of dictionary plays an important role in sparse representation. In this paper, a novel multiscale dictionary learning method is developed based on the multiscale structure of stationary wavelet. In the proposed multiscale dictionary learning method, the sub-dictionaries for all sub-bands of stationary wavelet are learned with a joint learning strategy. The advantages of the joint dictionary learning are that the learned multiscale dictionary can preserve the hierarchical structure of stationary wavelet, and can capture the intrinsic characteristics of images. To discuss the potential of the developed multiscale dictionary learning approach, sparse representation with the learned multiscale dictionary is applied into image fusion. The experimental results on various types of source images demonstrate that the multiscale dictionary based fusion method outperforms the single-scale dictionary based fusion methods, the mutiscale transform based fusion methods and the separated learned multiscale dictionary based fusion method.

Joint sparse coding based spatial pyramid matching for classification of color medical image

Although color medical images are important in clinical practice, they are usually converted to grayscale for further processing in pattern recognition, resulting in loss of rich color information. The sparse coding based linear spatial pyramid matching (ScSPM) and its variants are popular for grayscale image classification, but cannot extract color information. In this paper, we propose a joint sparse coding based SPM (JScSPM) method for the classification of color medical images. A joint dictionary can represent both the color information in each color channel and the correlation between channels. Consequently, the joint sparse codes calculated from a joint dictionary can carry color information, and therefore this method can easily transform a feature descriptor originally designed for grayscale images to a color descriptor. A color hepatocellular carcinoma histological image dataset was used to evaluate the performance of the proposed JScSPM algorithm. Experimental results show that JScSPM provides significant improvements as compared with the majority voting based ScSPM and the original ScSPM for color medical image classification.

Depth map Super-Resolution based on joint dictionary learning

Although Time-of-Flight (ToF) camera can provide real-time depth information from a real scene, the resolution of depth map captured by ToF camera is rather limited compared to HD color cameras, an...

Jointly Learning Visually Correlated Dictionaries for Large-Scale Visual Recognition Applications.

Learning discriminative dictionaries for image content representation plays a critical role in visual recognition. In this paper, we present a joint dictionary learning (JDL) algorithm which exploits the inter-category visual correlations to learn more discriminative dictionaries. Given a group of visually correlated categories, JDL simultaneously learns one common dictionary and multiple category-specific dictionaries to explicitly separate the shared visual atoms from the category-specific ones. The problem of JDL is formulated as a joint optimization with a discrimination promotion term according to the Fisher discrimination criterion. A visual tree method is developed to cluster a large number of categories into a set of disjoint groups, so that each of them contains a reasonable number of visually correlated categories. The process of image category clustering helps JDL to learn better dictionaries for classification by ensuring that the categories in the same group are of strong visual correlations. Also, it makes JDL to be computationally affordable in large-scale applications. Three classification schemes are adopted to make full use of the dictionaries learned by JDL for visual content representation in the task of image categorization. The effectiveness of the proposed algorithms has been evaluated using two image databases containing 17 and 1,000 categories, respectively.

Feature Denoising Using Joint Sparse Representation for In-Car Speech Recognition

We address reducing the mismatch between training and testing conditions for hands-free in-car speech recognition. It is well known that the distortions caused by background noise, channel effects, etc., are highly nonlinear in the log-spectral or cepstral domain. This letter introduces a joint sparse representation (JSR) to estimate the underlying clean feature vector from a noisy feature vector. Performing a joint dictionary learning by sharing the same representation coefficients, the proposed method intends to capture the complex relationships (or mapping functions) between clean and noisy speech. Speech recognition experiments on realistic in-car data demonstrate that the proposed method shows excellent recognition performance with a relative improvement of 39.4% compared with the “baseline” frontends.

Compressive high‐range‐resolution radar imaging using dynamic dictionaries

Compressive sensing theory suggests that accurate reconstruction of a signal can be achieved using its highly undersampled measurements, provided that the signal is sparse in an a priori known dictionary. For the range imaging problem in wideband radar, this dictionary is typically taken to be a DFT basis. However, since practical target scatterers do not lie exactly in the frequency lattice of the discrete Fourier transform (DFT) basis, there is always mismatch between the assumed DFT basis and the actual dictionary for sparsity. To address this, the authors consider the radar echo sparsifying dictionary as refinable and develop a compressive imaging approach using dynamic dictionaries. The approach treats the frequency gridding points as adjustable parameters of the sparsifying Fourier dictionary and achieves dynamical dictionary refinement via iterative optimisation of these parameters. To achieve joint image formation and dictionary refinement, the approach utilises the variational expectation-maximisation algorithm to iteratively perform a two-step process, that is, estimating sparse backscattering coefficients given a dictionary and then updating the dictionary to better fit the data sparsity model. The experimental results based on both synthetic and anechoic chamber data demonstrate that the approach improves the precision in range estimation and suppresses spurious spikes in the constructed profiles.

An improved joint dictionary training method for single image super resolution

PurposeThe purpose of this paper is to introduce an improved method for joint training of low‐ and high‐resolution dictionaries in the single image super resolution. With simulations, the proposed method is thereafter evaluated.Design/methodology/approachSparse representations of low‐resolution image patches are used to reconstruct the high‐resolution image patches with high resolution dictionary. By using different factors, the scheme weights the two dictionaries in the high‐ and low‐resolution spaces in the training process. It is reasonable to achieve better reconstructed images with more emphasis on the high‐resolution spaces.FindingsAn improved joint training algorithm based on K‐SVD is developed with flexible weight factors on dictionaries in the high‐ and low‐resolution spaces. From the experiment results, the proposed scheme outperforms the classic bicubic interpolation and neighbor‐embedding learning based method.Originality/valueBy using flexible weight factors in joint training of the dictionaries for super resolution, better reconstruction results can be achieved.

A Practical Compressed Sensing-Based Pan-Sharpening Method

High-resolution multispectral (HRM) images are widely used in many remote sensing applications. Using the pan-sharpening technique, a low-resolution multispectral (LRM) image and a high-resolution panchromatic (HRP) image can be fused to an HRM image. This letter proposes a new compressed sensing (CS)-based pan-sharpening method which views the image observation model as a measurement process in the CS theory and constructs a joint dictionary from LRM and HRP images in which the HRM is sparse. The novel joint dictionary makes the method practical in fusing real remote sensing images, and a tradeoff parameter is added in the image observation model to improve the results. The proposed algorithm is tested on simulated and real IKONOS images, and it results in improved image quality compared to other well-known methods in terms of both objective measurements and visual evaluation.

Coupled Dictionary Training for Image Super-Resolution

In this paper, we propose a novel coupled dictionary training method for single image super-resolution based on patchwise sparse recovery, where the learned couple dictionaries relate the low- and high-resolution image patch spaces via sparse representation. The learning process enforces that the sparse representation of a low-resolution image patch in terms of the low-resolution dictionary can well reconstruct its underlying high-resolution image patch with the dictionary in the highresolution image patch space. We model the learning problem as a bilevel optimization problem, where the optimization includes an 1-norm minimization problem in its constraints. Implicit differentiation is employed to calculate the desired gradient for stochastic gradient descent. We demonstrate that our coupled dictionary learning method can outperform the existing joint dictionary training method both quantitatively and qualitatively. Furthermore, for real applications, we speed up the algorithm approximately 10 times by learning a neural network model for fast sparse inference and selectively processing only those visually salient regions. Extensive experimental comparisons with stateof- the-art super-resolution algorithms validate the effectiveness of our proposed approach.