Dictionary Learning Approach Research Articles

A novel dictionary learning approach based on blind source separation basis and its application

The early recognition of wheel wear is an important task to the safe and efficient operation of a railway network. This article presents a new dictionary learning approach for wheel condition monitoring based on an adaptive parametric algorithm of blind source separation and extending K-means and singular value decomposition algorithm. Numerical simulations confirm the effectiveness of the proposed method. An experiment of wheel condition monitoring is conducted using a JD-1 wheel/rail simulation facility. Data calculation and theoretical analysis of wheel–rail contact dynamic show that the proposed method can adaptively learn and accurately identify wheel defects and verify the performance of the proposed method.

Robust coupled dictionary learning with ℓ1-norm coefficients transition constraint for noisy image super-resolution

Conventional coupled dictionary learning approaches are designed for noiseless image super-resolution (SR), but quite sensitive to noisy images. We find that the cause is the commonly used ℓF-norm coefficients transition term. In this paper, we propose a robust ℓ1-norm solution by introducing two sub-terms: LR coefficient sparsity constraint term and HR coefficient conversion term, which are able to prevent the noise transmission from noisy input to output. By incorporating our simple yet effective non-linear model inspired by auto-encoder, the proposed ℓ1-norm dictionary learning achieves a more accurate coefficients conversion. Moreover, to make the coefficients conversion more reliable in the iterative process, we bring the non-local self-similarity constraint to regularize the HR sparse coefficients updates. The improved sparse representation further enhances SR inference on both synthesized noisy and noiseless images. Using standard metrics, we show that results are significantly clearer than state-of-the-arts on noisy images and sharper on denoised images. In addition, experiments on real-world data further demonstrate the superiority of our method in practice.

An image fusion framework using novel dictionary based sparse representation

Image fusion is an important technique which aims to produce a synthetic result by leveraging the cross information available in the existing data. Sparse Representation (SR) is a powerful signal processing theory used in wide variety of applications like image denoising, compression and fusion. Construction of a proper dictionary with reduced computational efficiency is a major challenge in these applications. Owing to the above criterion, we propose a supervised dictionary learning approach for the fusion algorithm. Initially, gradient information is obtained for each patch of the training data set. Then, the edge strength and information content are measured for the gradient patches. A selection rule is finally employed to select the patches with better focus features for training the over complete dictionary. By the above process, the number of input patches for dictionary training is reduced to a greater extent. At the fusion step, the globally learned dictionary is used to represent the given set of source image patches. Experimental results with various source image pairs demonstrate that the proposed fusion framework gives better visual quality and competes with the existing methodologies quantitatively.

Adaptive group sparse representation in fetal echocardiogram segmentation

This paper present a novel group sparse representation model to segment four cardiac chambers in fetal echocardiograms. By incorporating the group reconstruction error, sparsity and distinctive term in a unified framework, the model is able to exploit the inherent structure of echocardiograms, constructing a discriminative group dictionary. A novel adaptive group dictionary learning approach is developed to obtain a compact dictionary with high atoms’ utilization and low complexity. With the learned dictionary, the reconstruction residue is employed to discriminate the initial location of four chambers. The local appearances are used to regionally refine the final contours. Extensive experiments have been conducted to evaluate the performance of our proposed AGDL and its application in the fetal echocardiogram segmentation. The results demonstrate both representation and discriminative power of our approach and its superior performances to other competing state-of-the-art sparse representation methods and general intensity models. Our approach is capable of learning a more compact and discriminative group dictionary, providing robust and accurate four-chamber segmentation results.

Multi-Kernel Low-Rank Dictionary Pair Learning for Multiple Features Based Image Classification

Dictionary learning (DL) is an effective feature learning technique, and has led to interesting results in many classification tasks. Recently, by combining DL with multiple kernel learning (which is a crucial and effective technique for combining different feature representation information), a few multi-kernel DL methods have been presented to solve the multiple feature representations based classification problem. However, how to improve the representation capability and discriminability of multi-kernel dictionary has not been well studied. In this paper, we propose a novel multi-kernel DL approach, named multi-kernel low-rank dictionary pair learning (MKLDPL). Specifically, MKLDPL jointly learns a kernel synthesis dictionary and a kernel analysis dictionary by exploiting the class label information. The learned synthesis and analysis dictionaries work together to implement the coding and reconstruction of samples in the kernel space. To enhance the discriminability of the learned multi-kernel dictionaries, MKLDPL imposes the low-rank regularization on the analysis dictionary, which can make samples from the same class have similar representations. We apply MKLDPL for multiple features based image classification task. Experimental results demonstrate the effectiveness of the proposed approach.

Learning Heterogeneous Dictionary Pair with Feature Projection Matrix for Pedestrian Video Retrieval via Single Query Image

Person re-identification (re-id) plays an important role in video surveillance and forensics applications. In many cases, person re-id needs to be conducted between image and video clip, e.g., re-identifying a suspect from large quantities of pedestrian videos given a single image of him. We call re-id in this scenario as image to video person re-id (IVPR). In practice, image and video are usually represented with different features, and there usually exist large variations between frames within each video. These factors make matching between image and video become a very challenging task. In this paper, we propose a joint feature projection matrix and heterogeneous dictionary pair learning (PHDL) approach for IVPR. Specifically, PHDL jointly learns an intra-video projection matrix and a pair of heterogeneous image and video dictionaries. With the learned projection matrix, the influence of variations within each video to the matching can be reduced. With the learned dictionary pair, the heterogeneous image and video features can be transformed into coding coefficients with the same dimension, such that the matching can be conducted using coding coefficients. Furthermore, to ensure that the obtained coding coefficients have favorable discriminability, PHDL designs a point-to-set coefficient discriminant term. Experiments on the public iLIDS-VID and PRID 2011 datasets demonstrate the effectiveness of the proposed approach.

Synthesis linear classifier based analysis dictionary learning for pattern classification

Dictionary learning approaches have been widely applied to solve pattern classification problems and have achieved promising performance. However, most of works aim to learn a discriminative synthesis dictionary and sparse coding coefficients for classification. Until recent years, analysis dictionary learning began to attract interest from researchers. In this paper, we present a novel discriminative analysis dictionary learning frame, named Synthesis Linear Classifier based Analysis Dictionary Learning (SLC-ADL). Firstly, we incorporate a synthesis-linear-classifier-based error term into the basic analysis dictionary learning model, whose classification performance is obviously improved by making full use of the label information. Then, we develop an alternating iterative algorithm to solve the new model and obtain closed-form solutions leading to pretty competitive running efficiency. What is more, we design three classification schemes by fully exploiting the synthesis linear classifier. Finally, extensive comparison experiments on scene categorization, object classification, action recognition and face recognition clearly verify the classification performance of the proposed algorithm.

Orthogonal Procrustes Analysis for Dictionary Learning in Sparse Linear Representation.

In the sparse representation model, the design of overcomplete dictionaries plays a key role for the effectiveness and applicability in different domains. Recent research has produced several dictionary learning approaches, being proven that dictionaries learnt by data examples significantly outperform structured ones, e.g. wavelet transforms. In this context, learning consists in adapting the dictionary atoms to a set of training signals in order to promote a sparse representation that minimizes the reconstruction error. Finding the best fitting dictionary remains a very difficult task, leaving the question still open. A well-established heuristic method for tackling this problem is an iterative alternating scheme, adopted for instance in the well-known K-SVD algorithm. Essentially, it consists in repeating two stages; the former promotes sparse coding of the training set and the latter adapts the dictionary to reduce the error. In this paper we present R-SVD, a new method that, while maintaining the alternating scheme, adopts the Orthogonal Procrustes analysis to update the dictionary atoms suitably arranged into groups. Comparative experiments on synthetic data prove the effectiveness of R-SVD with respect to well known dictionary learning algorithms such as K-SVD, ILS-DLA and the online method OSDL. Moreover, experiments on natural data such as ECG compression, EEG sparse representation, and image modeling confirm R-SVD’s robustness and wide applicability.

Super-Resolution Person Re-Identification With Semi-Coupled Low-Rank Discriminant Dictionary Learning.

Person re-identification has been widely studied due to its importance in surveillance and forensics applications. In practice, gallery images are high resolution (HR), while probe images are usually low resolution (LR) in the identification scenarios with large variation of illumination, weather, or quality of cameras. Person re-identification in this kind of scenarios, which we call super-resolution (SR) person re-identification, has not been well studied. In this paper, we propose a semi-coupled low-rank discriminant dictionary learning (SLD2L) approach for SR person re-identification task. With the HR and LR dictionary pair and mapping matrices learned from the features of HR and LR training images, SLD2L can convert the features of the LR probe images into HR features. To ensure that the converted features have favorable discriminative capability and the learned dictionaries can well characterize intrinsic feature spaces of the HR and LR images, we design a discriminant term and a low-rank regularization term for SLD2L. Moreover, considering that low resolution results in different degrees of loss for different types of visual appearance features, we propose a multi-view SLD2L (MVSLD2L) approach, which can learn the type-specific dictionary pair and mappings for each type of feature. Experimental results on multiple publicly available data sets demonstrate the effectiveness of our proposed approaches for the SR person re-identification task.

Device-Free Localization via Dictionary Learning with Difference of Convex Programming

In this paper, we consider a method to solve the device-free localization (DFL) problem that is able to detect spatial obstruction via wireless network. A dictionary learning approach with difference of convex (DC) programming and DC algorithm is proposed to indicate target location based on learning data. By measuring the variation in the received signal strength of the wireless links indicating the locations of the obstructions, the physical target in the monitoring area can be estimated through a learned dictionary. We show that the DFL problem can be efficiently formulated as a non-convex optimization problem. We adopt a penalty function called the minimax concave penalty, which possesses good properties in terms of seeking sparsity, and solve the non-convex optimization problem using DC programming. Furthermore, the localization accuracy achieved during the path-tracking task is further improved by the proposed tracking neighborhood rule. The rule provides a solution for increasing the localization accuracy under time-varying conditions generated by sampling channels of sensor networks under noisy conditions. The proposed approach is validated on a real-world dataset and has the potential to be adopted flexibly in DFL applications.

Sparse coding and dictionary learning with class-specific group sparsity

In recent years, sparse coding via dictionary learning has been widely used in many applications for exploiting sparsity patterns of data. For classification, useful sparsity patterns should have discrimination, which cannot be well achieved by standard sparse coding techniques. In this paper, we investigate structured sparse coding for obtaining discriminative class-specific group sparsity patterns in the context of classification. A structured dictionary learning approach for sparse coding is proposed by considering the \(\ell _{2,0}\) norm on each class of data. An efficient numerical algorithm with global convergence is developed for solving the related challenging \(\ell _{2,0}\) minimization problem. The learned dictionary is decomposed into class-specific dictionaries for the classification that is done according to the minimum reconstruction error among all the classes. For evaluation, the proposed method was applied to classifying both the synthetic data and real-world data. The experiments show the competitive performance of the proposed method in comparison with several existing discriminative sparse coding methods.

Multi-feature kernel discriminant dictionary learning for face recognition

The current study put forward a multi-feature kernel discriminant dictionary learning algorithm for face recognition. It was based on the supervised within-class-similar discriminative dictionary learning algorithm (SCDDL) we introduced previously. The proposed new algorithm was thus named as multi-feature kernel SCDDL (MKSCDDL). In contrast to the weighted combination or the constraint of representation coefficients for the feature combination used by some popular methods, MKSCDDL introduced the multiple kernel learning technique into the dictionary learning scheme. The experimental results on three large well-known face databases suggested that combination multiple features in MKSCDDL improved the recognition rate compared with SCDDL. In addition, adopting multiple kernel learning technique resulted in an excellent multi-feature dictionary learning approach when compared with some state-of-the-art multi-feature algorithms such as multiple kernel learning and multi-task joint sparse representation methods, indicating the effectiveness of the multiple kernel learning technique in the combination of multiple features for classification.

Automatic apical view classification of echocardiograms using a discriminative learning dictionary.

As part of striving towards fully automatic cardiac functional assessment of echocardiograms, automatic classification of their standard views is essential as a pre-processing stage. The similarity among three of the routinely acquired longitudinal scans: apical two-chamber (A2C), apical four-chamber (A4C) and apical long-axis (ALX), and the noise commonly inherent to these scans - make the classification a challenge. Here we introduce a multi-stage classification algorithm that employs spatio-temporal feature extraction (Cuboid Detector) and supervised dictionary learning (LC-KSVD) approaches to uniquely enhance the automatic recognition and classification accuracy of echocardiograms. The algorithm incorporates both discrimination and labelling information to allow a discriminative and sparse representation of each view. The advantage of the spatio-temporal feature extraction as compared to spatial processing is then validated. A set of 309 clinical clips (103 for each view), were labeled by 2 experts. A subset of 70 clips of each class was used as a training set and the rest as a test set. The recognition accuracies achieved were: 97%, 91% and 97% of A2C, A4C and ALX respectively, with average recognition rate of 95%. Thus, automatic classification of echocardiogram views seems promising, despite the inter-view similarity between the classes and intra-view variability among clips belonging to the same class.

Stratification of pseudoprogression and true progression of glioblastoma multiform based on longitudinal diffusion tensor imaging without segmentation.

Purpose:Pseudoprogression (PsP) can mimic true tumor progression (TTP) on magnetic resonance imaging in patients with glioblastoma multiform (GBM). The phenotypical similarity between PsP and TTP makes it a challenging task for physicians to distinguish these entities. So far, no approved biomarkers or computer-aided diagnosis systems have been used clinically for this purpose.Methods:To address this challenge, the authors developed an objective classification system for PsP and TTP based on longitudinal diffusion tensor imaging. A novel spatio-temporal discriminative dictionary learning scheme was proposed to differentiate PsP and TTP, thereby avoiding segmentation of the region of interest. The authors constructed a novel discriminative sparse matrix with the classification-oriented dictionary learning approach by excluding the shared features of two categories, so that the pooled features captured the subtle difference between PsP and TTP. The most discriminating features were then identified from the pooled features by their feature scoring system. Finally, the authors stratified patients with GBM into PsP and TTP by a support vector machine approach. Tenfold cross-validation (CV) and the area under the receiver operating characteristic (AUC) were used to assess the robustness of the developed system.Results:The average accuracy and AUC values after ten rounds of tenfold CV were 0.867 and 0.92, respectively. The authors also assessed the effects of different methods and factors (such as data types, pooling techniques, and dimensionality reduction approaches) on the performance of their classification system which obtained the best performance.Conclusions:The proposed objective classification system without segmentation achieved a desirable and reliable performance in differentiating PsP from TTP. Thus, the developed approach is expected to advance the clinical research and diagnosis of PsP and TTP.

Decomposing guided wavefields with dictionary learning

Ultrasonic guided waves have become a powerful tool for the structural health monitoring of large structures due to their low attenuation and high damage sensitivity. Yet, one significant challenge of working with guided waves is their complexity. Ultrasonic guided waves demonstrate strong multi-modal and dispersive characteristics and are often used in structures with many boundaries and reflecting elements. In prior work, compressive sensing theory and sparse recovery algorithms have been used to learn the multi-modal and dispersive properties of guided waves based on relatively simple wave models. The algorithms can predict and construct wavefields based on these learned properties. Furthermore, this predictive power can be used to detect and locate damage in a structure. However, these algorithms cannot yet predict and utilize reflections as well as other complexities (such as inhomogeneities) in a structure due to the simplicity of the current model. This paper presents a dictionary learning approach to learn this model directly from simulation or experimental data. This paper focuses on studying the models (i.e., dictionaries) created by dictionary learning under various simulated conditions. We demonstrate that we can use these dictionaries to accurately predict wavefields when waves are distorted by reflections, velocity inhomogeneities, and velocity anisotropy.

Multi-view Discriminant Dictionary Learning via Learning View-specific and Shared Structured Dictionaries for Image Classification

Recently, multi-view dictionary learning technique has attracted lots of research interest. Although some multi-view dictionary learning methods have been addressed, there exists much room for improvement. How to explore and utilize both the diversity and the useful correlation information of different views with dictionaries has not been well studied. In this paper, we propose a novel multi-view dictionary learning approach named multi-view discriminant dictionary learning via learning view-specific and shared structured dictionaries (MDVSD), which aims to learn a structured dictionary shared by all views and multiple view-specific structured dictionaries with each corresponding to a specific view. The shared dictionary is combined with each view-specific dictionary to represent data of the specific view. MDVSD makes the view-specific dictionaries corresponding to different views uncorrelated for effectively exploring the diversity of different views. Furthermore, we introduce structural uncorrelation into shared dictionary learning procedure, such that the useful correlation information of different views can be effectively exploited. Dictionary-atoms in shared and view-specific dictionaries have correspondence to class labels so that the learned dictionaries have favorable discriminant ability and the obtained reconstruction error is discriminative. Three widely used datasets are employed as test data. Experimental results demonstrate the effectiveness of the proposed approach.

A fast nonlocally centralized sparse representation algorithm for image denoising

The sparsity from self-similarity properties of natural images, which has received significant attention in the image processing community of researchers, is widely applied for image denoising. The recently proposed nonlocally centralized sparse representation (NCSR) algorithm that takes advantage of the sparse representations (SRs) and the nonlocal estimate of sparse coefficients (NESCs) has shown promising results with respect to noise reduction. Despite successful combination of the above two techniques, the iterative dictionary learning and the nonlocal estimate of unknown sparse coefficients make this algorithm computationally demanding, which largely limits its applicability in many applications. To address this problem, a fast version of the NCSR algorithm called FNCSR algorithm, which is based on pre-learned dictionary and adaptive parameter setting approaches, was proposed in this paper. Specifically, we adopted the same dictionary learning approach, i.e, the K-means and principal component analysis (PCA), with the NCSR algorithm to obtain a dictionary for each image in a selected image dataset including high-quality natural and texture images. Then we applied PNSR index to objectively assess the image quality of the reconstructed images using these dictionaries throughout the image dataset. The dictionary providing the best average reconstructed quality was selected as fixed dictionary, i.e., the pre-learned dictionary, for sparse coding throughout the iterative denoising process, which implies that it no longer requires dictionary learning procedure within the framework of the proposed FNCSR algorithm, resulting in greatly decreased execution time. In order to further improve computational efficiency, we employed quality-aware features and support vector regression (SVR) technique to build a fast noise level estimator (NLE) to estimate the noise level from a single noisy image. The parameters related to the NESC, i.e., the search window and the search step, which influences the computational performance of the NCSR algorithm strongly, were chosen automatically according to the estimated noise level. Compared to the original NCSR algorithm, these modifications lead to substantial benefits in computational efficiency (a performance gain of about 90% can be achieved) without sacrificing image quality too much (the largest decline is less than 0.55dB and 0.014 in terms of PSNR and SSIM indices). Compared with other state-of-the-art denoising algorithms, experimental results show that the proposed FNCNR algorithm also achieves comparable performance in terms of both quantitative measures and visual quality.

Object Recognition With an Elastic Net-Regularized Hierarchical MAX Model of the Visual Cortex

The human visual cortex has evolved to determine efficiently objects from within a scene. Hierarchical MAX (HMAX) is an object recognition model which has been inspired by the visual cortex, and sparse coding, which is a characteristic of neurons in the visual cortex, was previously integrated into the HMAX model for improved performance. In this study, in order to further enhance recognition accuracy, we have developed an elastic net-regularized dictionary learning approach for use in the HMAX model. We term this the En-HMAX model. With the En-HMAX model, we can exploit the sparsity-grouping tradeoff, such that correlated but informative features are preserved for object classification. Results show that the En-MAX model outperforms the original HMAX model in recognizing unseen objects by $\sim$ 40 $\%$ as well as the two special cases of the HMAX model, i.e., the least absolute shrinkage and selection operator (LASSO)-HMAX ( $\sim$ 19 $\%$ ) and Ridge-HMAX ( $\sim$ 9 $\%$ ) models.

Sparse Codes Auto-Extractor for Classification: A Joint Embedding and Dictionary Learning Framework for Representation

In this paper, we discuss the sparse codes auto-extractor based classification. A joint label consistent embedding and dictionary learning approach is proposed for delivering a linear sparse codes auto-extractor and a multi-class classifier by simultaneously minimizing the sparse reconstruction, discriminative sparse-code, code approximation and classification errors. The auto-extractor is characterized with a projection that bridges signals with sparse codes by learning special features from input signals for characterizing sparse codes. The classifier is trained based on extracted sparse codes directly. In our setting, the performance of the classifier depends on the discriminability of sparse codes, and the representation power of the extractor depends on the discriminability of input sparse codes, so we incorporate label information into the dictionary learning to enhance the discriminability of sparse codes. So, for inductive classification, our model forms an integration process from test signals to sparse codes and finally to assigned labels, which is essentially different from existing sparse coding based approaches that involve an extra sparse reconstruction with the trained dictionary for each test signal. Remarkable results are obtained by our model compared with other state-of-the-arts.

Dictionary Learning for Bioacoustics Monitoring with Applications to Species Classification

This paper deals with the application of the convolutive version of dictionary learning to analyze in-situ audio recordings for bio-acoustics monitoring. We propose an efficient approach for learning and using a sparse convolutive model to represent a collection of spectrograms. In this approach, we identify repeated bioacoustics patterns, e.g., bird syllables, as words and represent new spectrograms using these words. Moreover, we propose a supervised dictionary learning approach in the multiple-label setting to support multi-label classification of unlabeled spectrograms. Our approach relies on a random projection for reduced computational complexity. As a consequence, the non-negativity requirement on the dictionary words is relaxed. Furthermore, the proposed approach is well-suited for a collection of discontinuous spectrograms. We evaluate our approach on synthetic examples and on two real datasets consisting of multiple birds audio recordings. Bird syllable dictionary learning from a real-world dataset is demonstrated. Additionally, we successfully apply the approach to spectrogram denoising and species classification.