Discriminative Dictionary Learning Research Articles

Optimal discriminative feature and dictionary learning for image set classification

Image set classification has drawn increasing attention and it has been widely applied to many real-life domains. Due to the existence of multiple images in a set, which contain various view appearance changes, image set classification is a rather challenging task. One potential solution is to learn powerful representations from multiple images to decrease the intra-class diversity and enlarge the inter-class separation. In this paper, we propose an optimal discriminative feature and dictionary learning (ODFDL) method, which attempts to learn a feature mapping matrix and a dictionary such that in the mapped feature space the inter-class sparse reconstruction error of data is maximized and the intra-class sparse reconstruction error is minimized. This learning strategy enforces the learned sparse representations from image sets have large inter-class separation and small intra-class scatter. Furthermore, to better exploit the non-linear information of data from different image sets, we also present two non-linear ODFDL methods, termed Kernel-ODFDL and Hierarchy-ODFDL to further improve the classification performance. Experiments on five commonly used image sets exhibit that our approaches are comparable with many state-of-the-arts.

Predicting the progression of mild cognitive impairment to Alzheimer’s disease by longitudinal magnetic resonance imaging-based dictionary learning

ObjectiveEfficient prediction of the progression of mild cognitive impairment (MCI) to Alzheimer’s disease (AD) is important for the early intervention and management of AD. The aim of our study was to develop a longitudinal structural magnetic resonance imaging-based prediction system for MCI progression. MethodsA total of 164 MCI patients with longitudinal data were collected from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). After preprocessing, a discriminative dictionary learning framework was applied to differentiate MCI patches, avoiding the segmentation of regions of interest. Then, the proportion of patches classified as more severe atrophy patches in a patient was calculated as his or her feature to be input into a simple support vector machine. Finally, a new subject was predicted with fourfold cross-validation (CV), and the area under the receiver operating characteristic curve (AUC) was determined. ResultsThe average accuracy and AUC values after fourfold CV were 0.973 and 0.984, respectively. The effects of the data from one or two time points were also investigated. ConclusionThe proposed prediction system achieves desirable and reliable performance in predicting progression for MCI patients. Additionally, the prediction of MCI progression with longitudinal data was more effective and accurate. SignificanceThe developed scheme is expected to advance the clinical research and treatment of MCI patients.

Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning

We propose a joint subspace recovery and enhanced locality-based robust flexible label consistent dictionary learning method called Robust Flexible Discriminative Dictionary Learning (RFDDL). The RFDDL mainly improves the data representation and classification abilities by enhancing the robust property to sparse errors and encoding the locality, reconstruction error, and label consistency more accurately. First, for the robustness to noise and sparse errors in data and atoms, the RFDDL aims at recovering the underlying clean data and clean atom subspaces jointly, and then performs DL and encodes the locality in the recovered subspaces. Second, to enable the data sampled from a nonlinear manifold to be handled potentially and obtain the accurate reconstruction by avoiding the overfitting, the RFDDL minimizes the reconstruction error in a flexible manner. Third, to encode the label consistency accurately, the RFDDL involves a discriminative flexible sparse code error to encourage the coefficients to be soft. Fourth, to encode the locality well, the RFDDL defines the Laplacian matrix over recovered atoms, includes label information of atoms in terms of intra-class compactness and inter-class separation, and associates with group sparse codes and classifier to obtain the accurate discriminative locality-constrained coefficients and classifier. The extensive results on public databases show the effectiveness of our RFDDL.

Single-Channel sEMG Dictionary Learning Classification of Ingestive Behavior on Cows

Monitoring the ingestive behavior, especially rumination and eating, is important to assess animals’ health, welfare, needs, and to estimate production efficiency. In this paper, the myographic response of masticatory muscles was acquired from cows to directly access ingestive information (grazing and ruminating patterns) with high accuracy and time precision. Myographic signal processing is usually based on the same framework and optimal feature combinations have been extensively researched. However, recognition rates of hand-crafted feature based classifiers may be sensible to data deviation. In order to avoid the aforementioned issues and improve robustness, we propose an approach using sparse representation based classifiers which dismiss the need to select the best feature combination and classifier for the application. Also, we present a novel segmentation protocol to select chew related signal windows on cows. Four state-of-the-art multi-feature sets combined with linear discriminant analysis (LDA) were compared to a sparse representation based classifier called Fisher Discriminant Dictionary Learning (FDDL). Results suggest a significantly better performance of FDDL (p<0.05), achieving an accuracy rate of 90%, even in the presence of noise (0 – 20 dB), having a performance gain of 2.45% (SNR of 0 dB) in relation to the best hand-crafted feature set. When the noise was added just to the test set, all others classifiers remained under 75% (near 50%) for severe noise (0 – 10 dB), while FDDL stayed over 85%.

Optimal decision fusion using sparse representation‐based classifiers on monogenic‐signal dictionaries for SAR ATR

Sparse representation-based classification (SRC) possesses remarkable characteristics for application in synthetic aperture radar (SAR) automatic target recognition (ATR), for instance, inherent feature extraction and robustness to articulation, and so on. However, the performance of SRC is highly sensitive to parameters such as sufficient training samples, SAR images quality, and targets’ changing conditions in depression, pose, configuration, and so on. Unfortunately, the training sample resources for SAR ATR are often expensive and scarce. Further, the targets in SAR images, even with slightest changes in conditions, display mutable characteristics attributable to unique SAR image formation, and speckle noise corruption. To overcome these obstacles, this Letter proposes to establish several compact and complementary dictionaries using monogenic signal's components of SAR images and Fisher discriminative dictionary learning. Then, an optimal decision fusion (ODF) strategy is proposed, which utilises SRC and the latter dictionaries for robust SAR ATR. Compared with single classifiers or multiple parallel classifiers, the proposed ODF increases the accuracy of recognition, while at the same time, decreases the complexity of the system. The proposed methods have considerable features making them applicable in practical situations. Based on the experimental results, the proposed methods outperform state-of-the-art approaches.

Dictionary learning with low mutual coherence constraint

This paper presents efficient algorithms for learning low-coherence dictionaries. First, a new algorithm based on proximal methods is proposed to solve the dictionary learning (DL) problem regularized with the mutual coherence of dictionary. This is unlike the previous approaches that solve a regularized problem where an approximate incoherence promoting term, instead of the mutual coherence, is used to encourage low-coherency. Then, a new solver is proposed for constrained low-coherence DL problem, i.e., a DL problem with an explicit constraint on the mutual coherence of the dictionary. As opposed to current methods, which follow a suboptimal two-step approach, the new algorithm directly solves the associated DL problem. Using previous studies, convergence of the new schemes to critical points of the associated cost functions is also provided. Furthermore, it is shown that the proposed algorithms have lower iteration complexity than existing algorithms. Our simulation results on learning low-coherence dictionaries for natural image patches as well as image classification based on discriminative over-complete dictionary learning demonstrate the superiority of the proposed algorithms compared with the state-of-the-art method.

A Projective and Discriminative Dictionary Learning for High-Dimensional Process Monitoring With Industrial Applications

Data-driven process monitoring methods have attracted many attentions and gained wide applications. However, the real industrial process data are much more complex which is characterized by multimode, high dimensional, corrupted, and less labeled data. In order to eliminate these unfavourable factors simultaneously, a semisupervised robust projective and discriminative dictionary learning method is proposed. First, a semisupervised strategy is introduced to label unsupervised training data. Then, by utilizing low-rank and sparse features of raw data and outliers, a robust decomposition method is used to obtain clean data. After that, a simultaneously projective and discriminative model is proposed to extracting the feature of the low-rank clean data. Finally, the projection matrix and global dictionary, as well as the threshold are obtained through iterative dictionary learning. This hybrid framework provides a robust model for process monitoring and mode identification, and its efficiency is demonstrated by both synthetic examples and real industrial process cases.

Discriminative-shared dictionary learning for class-specific fabric texture characterization

In this paper we introduced a novel discriminative-shared dictionary learning (DSDL) model to explicitly extract a set of class-specific features as well as shared features for simultaneous fabric texture characterization. For the discriminative component, we imposed the constraints of minimizing inter-class correlation as well as maximizing intra-class correlation on them. For the shared component, we enforced a low-rank constraint and a similarity constraint on it. To demonstrate the characterization performance of the learned dictionary on multi-class fabric textures, we weaved eight fabric textures in the laboratory and then reconstructed each class of sample with each class of specific dictionary. Preliminary experiments with four-class samples demonstrate that the specific class of dictionary can only reconstruct the corresponding class of fabric texture, but has weak reconstructive ability for the rest class, and vice versa. In addition, we illustrated the contribution of shared dictionary by comparing the convergence rate of DSDL with that of Fisher Discriminative Dictionary Learning. Further, we developed a new indicator based on a similarity matrix for evaluating the discriminative of class-specific dictionaries, and we validated the effectiveness of the indicator by comparing the discriminative indicators of eight group sets. In general, the proposed DSDL model can effectively extract discriminative features and shared features simultaneously of multi-class fabric textures.

Decision fusion using virtual dictionary‐based sparse representation for robust SAR automatic target recognition

Sparse representation displays remarkable characteristics when applied to image processing and classification. The critical point in the success of sparse representation-based classification is to learn an authentic dictionary. The present study proposes a virtual dictionary-based sparse representation for automatic target recognition. Based on the properties of the synthetic aperture radar (SAR) images, some low complexity modules including adding speckle noise, histogram equalisation mapping, and bicubic interpolation are applied to construct some virtual compact dictionaries using Fisher discriminative dictionary learning. These dictionaries have different discriminative information on targets, which are used independently in several sparse representation-based classifiers. The reconstruction error vectors of the latter classifiers are then combined to recognise the target using decision fusion. Based on experimental results obtained drawing upon moving and stationary target acquisition and recognition data set, the proposed method presents the highest accuracy in classification reported yet in the literature. Furthermore, the procedure improves the recognition robustness against most commonly extended operating conditions, e.g. speckle noise corruption, depression angle variation and reduced training set. Accordingly, the current study claims a robust parallel method of high real-time ability in the target recognition of SAR images applicable to practical situations.

Infrared image super-resolution via discriminative dictionary and deep residual network

Image super-resolution successfully overcomes the limitation of infrared imaging system and satisfies the increasing demand for high-resolution infrared images. In this paper, we propose a novel infrared image super-resolution method via discriminative dictionary and deep residual network, which is benefited from the advantages of both compressive sensing and deep learning for more accurate reconstruction results. A discriminative dictionary learning method is introduced to explicitly learn a shared sub-dictionary with a set of cluster-specific sub-dictionaries from the training dataset to better characterize the specific representation of each image. Besides, multiple constraints are integrated to preserve the intrinsic relationship and structure information among images in the learned dictionary. Moreover, to further compensate for the limitation of dictionary representation capacity, a deep residual network is constructed to capture more high-frequency details of infrared images. Both quantitative and qualitative evaluations demonstrate that the proposed method could achieve favorable performances against other state-of-the-art methods.

Noise-robust image fusion with low-rank sparse decomposition guided by external patch prior

It is challenging to simultaneously achieve noise suppression and fine detail preservation in noisy image fusion. To address this challenge, we propose a novel strategy for noisy image fusion. Assuming that an image can be modeled as a superposition of low-rank and sparse (LR-and-S) components, we develop a novel discriminative dictionary learning algorithm to construct two dictionaries so as to decompose the input image into LR-and-S components. Specifically, to make dictionary possess discriminative power, we enforce spatial morphology constraint on each dictionary. Furthermore, we develop within-class consistency constraint by exploiting the similarity of low-rank components and impose it on the coding coefficients to further improve the discriminative power of the learned dictionary. In image decomposition, external patch prior and internal self-similarity prior of an image are exploited to build image decomposition model, based on which the latent subspace for fusion and recovery is estimated by minimizing rank-regularization of the subspace learned via clustering of similar patches. To construct different components of fused result, we use l1-norm maximization rule to fuse the decomposed components. Finally, the fused image is obtained by adding the fused components together. Experiments demonstrate that our method outperforms several state-of-the-art methods in terms of both objective quality assessment and subjective visual perception.

Nonconvex dictionary learning based visual tracking method

Focusing on the heavy decrease of object tracking performance induced by complex circumstances, an object tracking method based on nonconvex discriminative dictionary learning (NDDL) is proposed. Firstly, the object and background samples are acquired according to the temporal and spatial local correlation of objects. Since object and background samples have some common features, an inconsistent constraint is imposed on dictionaries to improve their robustness and discriminability. In what follows, a nonconvex minimax concave plus (MCP) function can be used to penalize sparse encoding matrices to avoid over-punishment via some convex relaxation methods. Based on the sparse representation (SR) theory, a NDDL model can be constructed, which can be tackled by majorization-minimization inexact augmented Lagrange multiplier (MM-IALM) optimization method to achieve better convergence. After obtaining the optimal discriminative dictionary, the reconstruction errors of all candidates are calculated to construct the object observation model. Finally, the object tracking is implemented accurately based on the Bayesian inference framework. Compared to the existing state-of-the-art trackers, simulation results show that the proposed tracker can improve the precision and success rate of the object tracking significantly in complex circumstances.

☆ - Discriminative dictionary learning algorithm based on sample diversity and locality of atoms for face recognition

Dictionary learning is one of the most important algorithms for face recognition. However, many dictionary learning algorithms for face recognition have the problems of small sample and weak discriminability. In this paper, a novel discriminative dictionary learning algorithm based on sample diversity and locality of atoms is proposed to solve the problems. The rational sample diversity is implemented by alternative samples and new error model to alleviate the small sample size problem. Moreover, locality can leads to sparsity and strong discriminability. In this paper, to enhance the dictionary discrimination and to reduce the influence of noise, the graph Laplacian matrix of atoms is used to keep the local information of the data. At the same, the relational theory is presented. A large number of experiments prove that the proposed algorithm can achieve more high performance than some state-of-the-art algorithms.

Deep locality‐sensitive discriminative dictionary learning for semantic video analysis

SummaryVideo semantic analysis (VSA) has received significant attention in the area of Machine Learning for some time now, particularly video surveillance applications with sparse representation and dictionary learning. Studies have shown that the duo has significantly impacted on the classification performance of video detection analysis. In VSA, the locality structure of video semantic data containing more discriminative information is very essential for classification. However, there has been modest feat by the current SR‐based approaches to fully utilize the discriminative information for high performance. Furthermore, similar coding outcomes are missing from current video features with the same video category. To handle these issues, we first propose an improved deep learning algorithm—locality deep convolutional neural network algorithm (LDCNN) to better extract salient features and obtain local information from semantic video. Second, we propose a novel DL method, called deep locality‐sensitive discriminative dictionary learning (DLSDDL) for VSA. In the proposed DLSDDL, a discriminant loss function for the video category based on sparse coding of sparse coefficients is introduced into the structure of the locality‐sensitive dictionary learning (LSDL) method. After solving the optimized dictionary, the sparse coefficients for the testing video feature samples are obtained, and then the classification result for video semantic is realized by reducing the error existing between the original and recreated samples. The experiment results show that the proposed DLSDDL technique considerably increases the efficiency of video semantic detection as against competing methods used in our experiment.

Discriminative dictionary learning based sparse representation classification for intelligent fault identification of planet bearings in wind turbine

Planet bearing fault identification is an attractive but challenging task in wind turbine condition monitoring and fault diagnosis. Traditional fault characteristic frequency identification based diagnostic strategies are not sufficient for reliable planet bearing fault detection, due to complex physical configurations and modulation characteristics in planetary drivetrains. In this paper, we propose a discriminative dictionary learning based sparse representation classification (DDL-SRC) framework for intelligent planet bearing fault identification. Our framework could learn a reconstructive and discriminative dictionary for signal sparse representation and an optimal linear classifier for classification tasks simultaneously, which bridges the gap between dictionary learning and classifier training in traditional SRC methods. Specifically, the optimization objective introduces a novel regularization term called ‘discriminative sparse codes error’ and incorporates it with the reconstruction error and classification error. Thus, the dictionary learned by our framework possesses not only the reconstructive power for sparse representation but also the discriminative power for classifier training. The optimization formulation is efficiently solved using K-SVD and orthogonal matching pursuit algorithms. The experiment validations have been conducted for demonstrating the effectiveness and superiority of the proposed DDL-SRC framework over the state-of-the-art dictionary learning based SRC and deep convolutional neural network methods for intelligent planet bearing fault identification.

Noise learning based discriminative dictionary learning algorithm for image classification

Dictionary learning is an efficient and effective method to preserve the label property in supervised learning for image classification. However, different noises in the image samples may cause unstable residuals during the training stage, which is particularly reflected in obtaining an inaccurate dictionary and inefficient utilization rate of label information. In order to fully exploit the supervised information for learning a discriminative dictionary, we propose an effective dictionary learning algorithm for designing structured dictionary where each atom related to a corresponding label. The proposed algorithm is implemented by alternating direction method of multipliers (ADMM) based on noise learning, where the noise is composed of interference signals and reconstruction residuals. In the training stage, we first adopt cross-label suppression method to enlarge the difference among the representations of different labels. Meanwhile, a mathematical operator of Laplacian matrix in spectral clustering named N-cut is also utilized to shorten the difference among the representations of same labels. In the testing stage, to take fully advantage of the learnt dictionary, two efficient classifiers of global coding and local coding are adopted in the denoising step respectively. Experiments are conducted in different datasets including face recognition, scene classification, object categorization, and dynamic texture categorization. Simulation results confirm our proposed method in terms of both classification performance and computational efficiency.

Tract Dictionary Learning for Fast and Robust Recognition of Fiber Bundles.

In this paper, we propose an efficient framework for parcellation of white matter tractograms using discriminative dictionary learning. Key to our framework is the learning of a compact dictionary for each fiber bundle so that the streamlines within the bundle can be sufficiently represented. Dictionaries for multiple bundles are combined for whole-brain tractogram representation. These dictionaries are learned jointly to encourage inter-bundle incoherence for discriminative power. The proposed method allows tractograms to be assigned to more than one bundle, catering to scenarios where tractograms cannot be clearly separated. Experiments on a bundle-labeled HCP dataset and an infant dataset highlight the ability of our framework in grouping streamlines into anatomically plausible bundles.

Structural Constraint and Discriminative Constraint Based Dictionary Pair Learning for Image Classification

The dictionary learning algorithm is an effective image classification algorithm. To tackle with the problems including noise, occlusion, small sample problem, and so on, many discriminative dictionary learning models have been proposed. However, the performance of these algorithms might not be satisfactory, the main reason is that some useful features are not effectively disclosed. For example, the coding coefficients of the samples have row sparsity consistency and the block structure property. Maintaining the inherent features of the samples (e.g. the block structure, local characteristics of atoms) is beneficial to enhance the discriminative ability of the model. In addition, a powerful discriminative constraint is also essential to improve the performance of image classification. In the paper, we propose a new structural constraint and discriminative constraint based dictionary pair learning for image classification. In the model, the L21 norm constraint is applied on the analysis sub-dictionary to ensure the row sparsity consistency of the coding coefficients as much as possible. A new discriminative constraint is designed to enforce the representation matrix to be more discriminative, which can be acquired by minimizing the intra-class scatter and maximizing the inter-class scatter of the samples. Besides, we define a new atomic locality constraint, which forces the atoms to preserve the structure information of the samples. Finally, seven benchmark data sets are selected to evaluate the performance of the proposed method in comparison with popular methods. The experimental results outperform the state-of-the-art methods, which demonstrates the efficacy of the proposed model.

Network Dissensus via Distributed ADMM

As machine learning implementations move towards the network edge, distributed optimization algorithms are becoming increasingly relevant. Common approaches build upon the consensus framework to enforce cooperation and consistency among the nodes. In many applications however, the inter-node relationships are better modeled via antagonistic or dissensual constraints. These relationships can generally be incorporated via non-convex constraints or penalty functions, and the resulting formulations are flexible enough to subsume a wide variety of classification and discrimination problems. This work develops a general-purpose ADMM algorithm for distributed optimization with dissensus constraints. The formulation is generalized to incorporate both consensus and dissensus relationships. The non-convex constraints are handled via appropriate first-order approximations and convergence to a stationary point is established. The proposed algorithm is tested on various synthetic as well as real-world data sets. In particular, we focus on the discriminative dictionary learning problem, where the goal is to learn class-specific dictionaries usable for both reconstruction and discrimination tasks. Extensive tests over seizure detection, activity recognition, and indoor localization datasets demonstrate the efficacy of the proposed approach.

Hierarchical Discriminative Deep Dictionary Learning

In deep dictionary learning multiple dictionaries are learned based on information at various levels of abstraction. We propose a novel hierarchical discriminative dictionary learning layer embedded within a neural network with an image classification objective. Discrimination is induced in the learned synthesis dictionaries at multiple hierarchical levels in a simple way by a one-hot-code representation of the class labels during the training backward pass. In addition, local sparse representation objectives are approximated during the forward pass, introducing local regularization. We evaluate our proposal on five known datasets and we either outperform state-of-the-art methods or achieve competitive classification results.