Discriminative Dictionary Learning Research Articles

A novel multi-layer discriminative dictionary learning approach for image classification

Discriminative dictionary learning (DDL) has been confirmed to be effective for image classification. However, existing DDL approaches often fail to extract deep hierarchical information due to the single-layer dictionary learning framework. Moreover, they overlook the atoms-label information in the dictionary, leading to reduced feature distinctiveness and lower classification accuracy. To overcome the above problem, a novel DDL method, called the Multi-layer local constraint and label embedding dictionary learning (M-LCLEDL), is proposed. Specifically, the novel multi-layer DDL framework, which is formed by stacking the DL process one by one, is designed to learn the deep hierarchical and nonlinear features. The layer-by-layer stacking of the DL process in the multi-layer DDL framework allows for the elimination of redundant and interfering features. This step-by-step elimination process enhances the stability and robustness of the framework. Additionally, to leverage the label information carried by the labeled training samples, atoms with label embedding and locality structure are introduced. The proposed approach includes a fast iteration strategy for efficient optimization. Experimental results demonstrate that the approach is relatively insensitive to dictionary size, achieving promising performance and greater stability compared to most DDL-based image classification approaches.

Fisher discrimination multiple kernel dictionary learning for robust identification of nonlinear features in machinery health monitoring

In sparse representation-based classification, Fisher discrimination dictionary learning (FDDL) has attracted widespread attention due to its advantages such as fewer parameters and good dictionary discriminability. However, due to its linear nature, it is difficult to handle nonlinear features. Kernel-based nonlinear transformation enables dictionary learning to capture nonlinear features embedded in samples, but traditional single-kernel methods have limited performance. In this paper, a Fisher discrimination multiple kernel dictionary learning (FDMKDL) method is proposed, in which Fisher discriminative criterion is imposed on both high-dimensional samples and coding coefficients. Specifically, we derive the kernel version of FDDL, i.e., Fisher discrimination kernel dictionary learning (FDKDL) to learn nonlinear features and promote dictionary discriminability. Meanwhile, to avoid the problem of poor adaptability and possible manual selection caused by single-kernel methods, we further derive a flexible multiple kernel learning (MKL) framework, which utilizes the complementary information of multiple kernel functions to adaptively obtain the optimal weights and synthesize a discriminative kernel space. Finally, FDMKDL combines FDKDL with this MKL framework to obtain a more discriminative dictionary. Experimental evaluations conducted on two datasets demonstrate the effectiveness and robustness of the proposed FDMKDL method in learning nonlinear features for machinery health monitoring when compared to several state-of-the-art methods.

Discriminative dictionary learning for nonnegative representation based classification

Representation based classification (RC) algorithms have been successfully applied to pattern classification. However, most existing RC algorithms are not robust to bad training samples, since they ignore learning more discriminative dictionary atoms and instead directly use training samples as dictionary atoms. In addition, in order to improve expression ability and recognition accuracy, RC algorithms often need to expand the number of dictionary atoms, resulting in a surge in storage and computing costs. To obtain a more discriminative and compact dictionary, this study proposes discriminative dictionary learning for nonnegative representation based classification (DDLNRC). Specifically, in the DDLNRC, this paper utilizes nonnegative constraint to obtain a nonnegative representation for each training sample on the dictionary. In the dictionary learning stage, for a training sample, the DDLNRC minimizes the intra-class reconstruction error of the training sample and simultaneously enlarges the distance between the training sample and the atom that has the most influence on inter-class reconstruction error. Experiments demonstrate the effectiveness of the DDLNRC. Combined with the deep neural network features, it also can achieve higher accuracy than using the Softmax.

Background covariance discriminative dictionary learning for hyperspectral target detection

Hyperspectral target detection (HTD) aims to identifying targets within a hyperspectral image (HSI) based on provided target spectra. In the current HTD field, representation-based detectors have attracted much attention. However, there are two prominent challenges that are particularly noteworthy. First, the background class encompasses diverse land covers, making its accurate representation challenging. Second, the detection ability can be significantly influenced by the abnormalities and noise in HSI. To tackle these concerns, we propose a novel background covariance discriminative dictionary learning (BCDDL) model for HTD. To enhance the background representation ability and overcome the sparse noise, we combine the dictionary learning with spectral covariance descriptors and undertake background reconstruction in regional scale. Specifically, the input HSI is pre-processed into superpixels, the spectral covariance of each superpixel is used to provide a compact and flexible description of local regional statistical properties. Further, a novel spatial clustering-based dictionary learning method is proposed to learn the background discriminative covariances dictionary. The collaborative representation model within symmetric positive definite (SPD) manifold is utilized to reconstruct background region and get the residual. By merging the background residual with pixel-wise target reconstruction residual, we derive final detection output. Comprehensive experiments on two public hyperspectral datasets and two novel GaoFen-5 datasets demonstrate the superiority of our BCDDL approach over 10 state-of-the-art methods, especially in terms of suppressing background.

Structured collaborative sparse dictionary learning for monitoring of multimode processes

In this paper, a novel structured collaborative sparse dictionary learning approach is proposed to improve the monitoring performance of discriminative dictionary learning for multimode processes. The mode discriminability and data reconstruction are first balanced by decomposing the dictionary coefficients into between- and within-class parts and introducing a within-class self-expression regularization term. A weight vector of between-class coefficients is subsequently exploited for accurate mode identification of data that falls into the overlapping regions between different class distributions. Moreover, in order to pinpoint the fault variables, a scalable fault isolation method is developed which imposes a constraint of statistical control limit and introduces the ℓ1/ℓ2,0-structured sparsity regularization terms. The mode identification capability of the proposed method is proved theoretically by Theorems 1 and 2, and a lower-bound magnitude is provided by Theorem 3 for fault isolation. Finally, extensive experiments conducted in the numerical and industrial process demonstrate that our proposed method outperforms some state-of-the-art methods.

Multimodal classification of Alzheimer's disease and mild cognitive impairment using custom MKSCDDL kernel over CNN with transparent decision-making for explainable diagnosis

The study presents an innovative diagnostic framework that synergises Convolutional Neural Networks (CNNs) with a Multi-feature Kernel Supervised within-class-similar Discriminative Dictionary Learning (MKSCDDL). This integrative methodology is designed to facilitate the precise classification of individuals into categories of Alzheimer's Disease, Mild Cognitive Impairment (MCI), and Cognitively Normal (CN) statuses while also discerning the nuanced phases within the MCI spectrum. Our approach is distinguished by its robustness and interpretability, offering clinicians an exceptionally transparent tool for diagnosis and therapeutic strategy formulation. We use scandent decision trees to deal with the unpredictability and complexity of neuroimaging data. Considering that different people's brain scans are different, this enables the model to make more detailed individualised assessments and explains how the algorithm illuminates the specific neuroanatomical regions that are indicative of cognitive impairment. This explanation is beneficial for clinicians because it gives them concrete ideas for early intervention and targeted care. The empirical review of our model shows that it makes diagnoses with a level of accuracy that is unmatched, with a classification efficacy of 98.27%. This shows that the model is good at finding important parts of the brain that may be damaged by cognitive diseases.

Interactive and discriminative analysis dictionary learning for image classification

Interactive and discriminative analysis dictionary learning for image classification

Discriminative Dictionary Learning Using Penalized Rank-1 Approximation for Breast Cancer Classification With Imbalanced Dataset

Discriminative Dictionary Learning Using Penalized Rank-1 Approximation for Breast Cancer Classification With Imbalanced Dataset

Wasserstein Discriminant Dictionary Learning for Graph Representation.

Mining discriminative graph topological information plays an important role in promoting graph representation ability. However, it suffers from two main issues: (1) the difficulty/complexity of computing global inter-class/intra-class scatters, commonly related to mean and covariance of graph samples, for discriminant learning; (2) the huge complexity and variety of graph topological structure that is rather challenging to robustly characterize. In this paper, we propose the Wasserstein Discriminant Dictionary Learning (WDDL) framework to achieve discriminant learning on graphs with robust graph topology modeling, and hence facilitate graph-based pattern analysis tasks. Considering the difficulty of calculating global inter-class/intra-class scatters, a reference set of graphs (aka graph dictionary) is first constructed by generating representative graph samples (aka graph keys) with expressive topological structure. Then, a Wasserstein Graph Representation (WGR) process is proposed to project input graphs into a succinct dictionary space through the graph dictionary lookup. To further achieve discriminant graph learning, a Wasserstein discriminant loss (WD-loss) is defined on the graph dictionary, in which the graph keys are optimizable, to make the intra-class keys more compact and inter-class keys more dispersed. Hence, the calculation of global Wasserstein metric (W-metric) centers can be bypassed. For sophisticated topology mining in the WGR process, a joint-Wasserstein graph embedding module is constructed to model both between-node and between-edge relationships across inputs and graph keys by encapsulating both the Wasserstein metric (between cross-graph nodes) and proposed novel Kron-Gromov-Wasserstein (KGW) metric (between cross-graph adjacencies). Specifically, the KGW-metric comprehensively characterizes the cross-graph connection patterns with the Kronecker operation, then adaptively captures those salient patterns through connection pooling. To evaluate the proposed framework, we study two graph-based pattern analysis problems, i.e. graph classification and cross-modal retrieval, with the graph dictionary flexibly adjusted to cater to these two tasks. Extensive experiments are conducted to comprehensively compare with existing advanced methods, as well as dissect the critical component of our proposed architecture. The experimental results validate the effectiveness of the WDDL framework.

Sparse Representations Optimization with Coupled Bayesian Dictionary and Dictionary Classifier for Efficient Classification

Among the numerous techniques followed to learn a linear classifier through the discriminative dictionary and sparse representations learning of signals, the techniques to learn a nonparametric Bayesian classifier jointly and discriminately with the dictionary and the corresponding sparse representations have drawn considerable attention from researchers. These techniques jointly learn two sets of sparse representations, one for the training samples over the dictionary and the other for the corresponding labels over the dictionary classifier. At the prediction stage, the representations of the test samples computed over the learned dictionary do not truly represent the corresponding labels, exposing weakness in the joint learning claim of these techniques. We mitigate this problem and strengthen the joint by learning a set of weights over the dictionary to represent the training data and further optimizing the same weights over the dictionary classifier to represent the labels of the corresponding classes of the training data. Now, at the prediction stage, the representation weights of the test samples computed over the learned dictionary also represent the labels of the corresponding classes of the test samples, resulting in the accurate reconstruction of the labels of the classes by the learned dictionary classifier. Overall, a reduction in the size of the Bayesian model’s parameters also improves training time. We analytically and nonparametrically derived the posterior conditional probabilities of the model from the overall joint probability of the model using Bayes’ theorem. We used the Gibbs sampler to solve the joint probability of the model using the derived conditional probabilities, which also supports our claim of efficient optimization of the coupled/joint dictionaries and the sparse representation parameters. We demonstrated the effectiveness of our approach through experiments on the standard datasets, i.e., the Extended YaleB and AR face databases for face recognition, Caltech-101 and Fifteen Scene Category databases for categorization, and UCF sports action database for action recognition. We compared the results with the state-of-the-art methods in the area. The classification accuracies, i.e., 93.25%, 89.27%, 94.81%, 98.10%, and 95.00%, of our approach on the datasets have increases of 0.5 to 2% on average. The overall average error margin of the confidence intervals in our approach is 0.24 compared with the second-best approach, JBDC, for which it is 0.34. The AUC–ROC scores of our approach are 0.98 and 0.992, which are better than those of others, i.e., 0.960 and 0.98, respectively. Our approach is also computationally efficient.

Discriminative semisupervised dictionary learning method with graph embedding and pairwise constraints for crop disease image recognition

Crop diseases are serious biological disasters that affect agricultural production and ecological environment safety. Accurate detection and identification of disease types can effectively reduce the harm caused by diseases. However, there are various types of crop diseases and labeling them consumes a lot of labor and material resources. A discriminative semisupervised dictionary learning method with graph embedding and pairwise constraints (DSDL-GEPC) is proposed in this study. DSDL-GEPC adopts semisupervised dictionary learning as the baseline model, which combines the reconstruction error of all training samples and the classification error of labeled training samples. DSDL-GEPC consists of three learning terms. The graph embedding term is constructed based on dictionary atoms to obtain global geometric structural information of data samples. Using the pairwise constraint information of similar/heterogeneous images, the pairwise constraints term is constructed to enhance the dictionary's discriminability in terms of within-class similarity and between-class separability. In addition, the relaxed low-rank classification term is constructed, which introduces the ε-draggings technique and low-rank constraint on the relaxed label matrix to learn a discriminative classifier. Experiments are performed to compare the classification performance of DSDL-GEPC on the PlantVillage dataset. The results validate the effectiveness of DSDL-GEPC for crop disease image recognition.

Double-constrained structured discriminant analysis-synthesis dictionary pair learning for pattern classification

Double-constrained structured discriminant analysis-synthesis dictionary pair learning for pattern classification

Class-oriented discriminative twin reconstructions dictionary pair learning for visual recognition

Class-oriented discriminative twin reconstructions dictionary pair learning for visual recognition

Discriminative analysis dictionary learning with adaptively ordinal locality preserving

Dictionary learning has found broad applications in signal and image processing. By adding constraints to the traditional dictionary learning model, dictionaries with discriminative capability can be obtained which can deal with the task of image classification. The Discriminative Convolutional Analysis Dictionary Learning (DCADL) algorithm proposed recently has achieved promising results with low computational complexity. However, DCADL is still limited in classification performance because of the lack of constraints on dictionary structures. To solve this problem, this study introduces an adaptively ordinal locality preserving (AOLP) term to the original model of DCADL to further improve the classification performance. With the AOLP term, the distance ranking in the neighborhood of each atom can be preserved, which can improve the discrimination of coding coefficients. In addition, a linear classifier for the classification of coding coefficients is trained along with the dictionary. A new method is designed specifically to solve the optimization problem corresponding to the proposed model. Experiments are performed on several commonly used datasets to show the promising results of the proposed algorithm in classification performance and computational efficiency.

Jointly projection and graph-regularization coupled discriminative dictionary learning for image classification

Jointly projection and graph-regularization coupled discriminative dictionary learning for image classification

A new weighted extreme learning machine based on elastic net regularization embedded exponential regularized discriminative dictionary learning for image classification

It is well known that discriminative sparse representation can significantly improve the performance of image classification. However, there remain several tricky issues to be addressed due to the unsatisfied performance and high time consumption. In this paper, a novel classification framework called weighted extreme learning machine exponential regularized discriminative dictionary learning (WELM-ERDDL) is proposed to address these issues. The main contributions of this paper include (1) the WELM is embedded with ERDDL via exponential regularized linear discriminative analysis (ERLDA) for feature mappings while enabling nonlinear and diverse feature representation; (2) in the ELM learning process, the elastic net regularization is utilized to optimize more robust and meaningful output weights; (3) an effective weight update rule is designed for WELM. To verify the effectiveness of the proposed method, several experiments are conducted on real-world image classification databases. The results show that the proposed WELM-ERDDL framework is even more efficient than other state-of-the-art algorithms in general.

Semi-supervised Discriminative Projective Dictionary Pair Learning and Its Application to Industrial Process

Industrial process data have the characteristics of less label, multimode, high dimension, containing noise, and mixing with outliers, which increase the difficulty of mode identification and anomaly detection in process monitoring using limited labeled data. In this article, to address the effect of these adverse factors, a semi-supervised discriminative projective dictionary pair learning (SSDP-DPL) is proposed for industrial process monitoring. First, a semi-supervised dictionary pair learning framework with a class estimation regularization term is developed to address the problem of lacking labeled data. Based on unlabeled data and their reconstruction errors, the class estimation regularization term is designed to obtain a discriminative extended synthetical dictionary, mining the hidden discriminative information in unlabeled data and reducing the impact of incorrect class estimation. Second, a sparse constraint is added to the analytical dictionary to enhance the robustness of projective dictionary pair. Then, a class discriminative function term is designed to improve the discrimination of dictionary pair, which ensures the intraclass compactness and interclass separation of coding space. Finally, the synthetical dictionary, the analytical dictionary, and the control threshold are obtained by iterating the dictionary pair to a process monitoring model for anomaly detection and mode identification. The effectiveness of the proposed process monitoring method is verified by the continuous stirred tank heater process and Tennessee Eastman benchmark tests. The proposed method is then applied to a real-world aluminum electrolysis industrial process. Experimental results demonstrate the superiority of our SSDP-DPL in contrast to other existing methods.

Joint structured constraint discriminant analysis dictionary learning for pattern classification

Analysis dictionary learning (DL) has been successfully applied to the field of pattern classification. However, it is still a challenge to utilize the local structure information and the class information of samples to improve the discrimination capability of analysis dictionary. We proposed a joint structured constraint discriminant analysis DL (ADL) method (JSCDADL) to learn a structured discriminant analysis dictionary by combining the local structure information and the structured information of samples. Specifically, we first designed an adaptive local structure preserving term (ALSPT) to improve the discrimination capability of analysis dictionary. It adaptively transmits the local structure information of samples to analysis dictionary, which ensures that the same class of samples has similar sparse codes under the action of the analysis dictionary. Then, we designed a discriminative sparse coding error term that forces the coding coefficient matrix to have the desired block diagonal structure. To further enhance the discrimination capability of analysis dictionary, we designed an analysis dictionary combination term by constantly approximating the two analysis dictionaries learned to obtain an analysis dictionary with the local structure information and the structured information of samples. Moreover, we designed an effective iterative algorithm to solve the optimization problem of JSCDADL. Extensive experimental results on six datasets demonstrate that JSCDADL can achieve satisfactory classification performance compared with some state-of-the-art methods.

A locally unsupervised hybrid learning method for removing environmental effects under different measurement periods

Environmental effects induce deceptive variability in unlabeled vibration data for structural health monitoring (SHM). Although unsupervised learning is an effective solution to this issue, some new challenges such as the size of training data in different measurement periods and the type of learning between local and global frameworks seriously affect overall performance of this technique. To tackle these issues, we propose a locally unsupervised hybrid learning method based on an innovative discriminative reconstruction-based dictionary learning (DRDL) algorithm. The proposed method initially uses a Gaussian mixture model to provide local information for the DRDL algorithm by clustering entire training data into local subsets. Subsequently, this algorithm computes sub-dictionaries and sparse coefficients to reconstruct local training subsets. Using these subsets, an anomaly detector developed from the Mahalanobis-squared distance is used to determine damage indices for SHM. Real data from two bridges are incorporated to verify the proposed method with some comparisons.

Standard simplex induced clustering with hierarchical deep dictionary learning

Clustering remains a fundamental but crucial task on unsupervised pattern recognition. A recent novel deep dictionary learning system where hierarchical discriminative dictionary learning layers are embedded within a neural network showed either competitive or state-of-the-art results for image classification. We now propose a new method for image clustering, extending hierarchical deep dictionary learning to the unsupervised learning setting. Clustering is induced in the neural network in a simple way by requesting vector outputs associated to categories to approximately lie in the corners of a standard simplex and by approximating a given expected value on such vectors. This is carried out in a scalable way due to the batch-wise updates as clustering is learned in the deep system. We evaluate our proposal on four benchmark datasets and we either achieve competitive results or outperform state-of-the-art clustering methods. Under the unbalanced scenario clustering is known to be more challenging. For this our method additionally considers Bayesian updates to automatically learn the correct cluster proportions for a small number of clusters.