Discriminative Dictionary Research Articles

Semi-supervised Discriminative Projective Dictionary Pair Learning and Its Application to Industrial Process

Industrial process data have the characteristics of less label, multimode, high dimension, containing noise, and mixing with outliers, which increase the difficulty of mode identification and anomaly detection in process monitoring using limited labeled data. In this article, to address the effect of these adverse factors, a semi-supervised discriminative projective dictionary pair learning (SSDP-DPL) is proposed for industrial process monitoring. First, a semi-supervised dictionary pair learning framework with a class estimation regularization term is developed to address the problem of lacking labeled data. Based on unlabeled data and their reconstruction errors, the class estimation regularization term is designed to obtain a discriminative extended synthetical dictionary, mining the hidden discriminative information in unlabeled data and reducing the impact of incorrect class estimation. Second, a sparse constraint is added to the analytical dictionary to enhance the robustness of projective dictionary pair. Then, a class discriminative function term is designed to improve the discrimination of dictionary pair, which ensures the intraclass compactness and interclass separation of coding space. Finally, the synthetical dictionary, the analytical dictionary, and the control threshold are obtained by iterating the dictionary pair to a process monitoring model for anomaly detection and mode identification. The effectiveness of the proposed process monitoring method is verified by the continuous stirred tank heater process and Tennessee Eastman benchmark tests. The proposed method is then applied to a real-world aluminum electrolysis industrial process. Experimental results demonstrate the superiority of our SSDP-DPL in contrast to other existing methods.

Supervised poststack 3D seismic data classification via multiscale and multilabel consistent feature reduction

Following the advancement of machine learning-based seismic feature classification techniques for complex reservoirs, the acquisition and analysis of reliable seismic samples involved in seismic facies analysis and network-based inversion have emerged as a current research hotspot in the field of intelligent seismic processing. Many investigations focus on the improvement of model classification algorithms and neural networks. However, creating and collecting labels for massive seismic data are highly time-consuming and laborious, and suffer from sample unreliability and category imbalance in the case of small-sample labels. To address such problems, a multiscale and multilabel consistent principal component analysis-linear discriminant analysis (PCA-LDA) algorithm to learn a robust feature discriminative dictionary for classification is presented. In addition to the automatic use of multilabels from well logs and core analysis, we have associated multiscale with well trajectory locations to enrich sample information and enhance the reliability of the samples during 3D sample acquisition. More specifically, we begin by proposing an approach for the automatic collection of multiscale multilabel 3D poststack seismic samples along the well track. Next, the multilabel sequence in the scan window is fed into the Boyer-Moore majority vote algorithm for sample segmentation, which constructs multilabel hierarchies for each sample. Then to enhance the model training bias due to small-sample label imbalance, we develop a novel label-shuffling balanced strategy, which obtains a complete database by filling random unduplicated augmented training samples (spatial and frequency-domain augmentation operations). Finally, the linear robustness decision-making space of PCA-LDA is obtained using the feature mapping space of PCA, as well as its visual representation. Experimental results on synthetic and field seismic data demonstrate that robust feature extraction with a trustworthy and complete multiscale and multilabel sample database increases classification accuracy.

Joint Augmented and Compressed Dictionaries for Robust Image Classification

Dictionary-based Classification (DC) has been a promising learning theory in multimedia computing. Previous studies focused on learning a discriminative dictionary as well as the sparsest representation based on the dictionary, to cope with the complex conditions in real-world applications. However, robustness by learning only one single dictionary is far from the optimal level. What is worse, it cannot take advantage of the available techniques proven in modern machine learning, like data augmentation, to mitigate the same problem. In this work, we propose a novel method that utilizes joint Augmented and Compressed Dictionaries for Robust Dictionary-based Classification (ACD-RDC). For optimization under the noise model introduced by real-world conditions, the objective function of ACD-RDC incorporates only two simple, but well-designed constraints, including one enhanced sparsity constraint by the general data augmentation, which requires less case-by-case and sophisticated tuning, and another discriminative constraint solved by a jointly learned dictionary. The optimization of the objective function is then deduced theoretically to an approximate linear problem. The sparsity and discrimination enhanced by data augmentation guarantees the robustness for image classification under various conditions, which constructs the first positive case using data augmentation to obtain robust dictionary-based classification. Numerous experiments have been conducted on popular facial and object image datasets. The results demonstrate that ACD-RDC obtains more promising classification on diversely collected images than the current dictionary-based classification methods. ACD-RDC is also confirmed to be a state-of-the-art classification method when using deep features as inputs.

Joint structured constraint discriminant analysis dictionary learning for pattern classification

Analysis dictionary learning (DL) has been successfully applied to the field of pattern classification. However, it is still a challenge to utilize the local structure information and the class information of samples to improve the discrimination capability of analysis dictionary. We proposed a joint structured constraint discriminant analysis DL (ADL) method (JSCDADL) to learn a structured discriminant analysis dictionary by combining the local structure information and the structured information of samples. Specifically, we first designed an adaptive local structure preserving term (ALSPT) to improve the discrimination capability of analysis dictionary. It adaptively transmits the local structure information of samples to analysis dictionary, which ensures that the same class of samples has similar sparse codes under the action of the analysis dictionary. Then, we designed a discriminative sparse coding error term that forces the coding coefficient matrix to have the desired block diagonal structure. To further enhance the discrimination capability of analysis dictionary, we designed an analysis dictionary combination term by constantly approximating the two analysis dictionaries learned to obtain an analysis dictionary with the local structure information and the structured information of samples. Moreover, we designed an effective iterative algorithm to solve the optimization problem of JSCDADL. Extensive experimental results on six datasets demonstrate that JSCDADL can achieve satisfactory classification performance compared with some state-of-the-art methods.

A locally unsupervised hybrid learning method for removing environmental effects under different measurement periods

Environmental effects induce deceptive variability in unlabeled vibration data for structural health monitoring (SHM). Although unsupervised learning is an effective solution to this issue, some new challenges such as the size of training data in different measurement periods and the type of learning between local and global frameworks seriously affect overall performance of this technique. To tackle these issues, we propose a locally unsupervised hybrid learning method based on an innovative discriminative reconstruction-based dictionary learning (DRDL) algorithm. The proposed method initially uses a Gaussian mixture model to provide local information for the DRDL algorithm by clustering entire training data into local subsets. Subsequently, this algorithm computes sub-dictionaries and sparse coefficients to reconstruct local training subsets. Using these subsets, an anomaly detector developed from the Mahalanobis-squared distance is used to determine damage indices for SHM. Real data from two bridges are incorporated to verify the proposed method with some comparisons.

Standard simplex induced clustering with hierarchical deep dictionary learning

Clustering remains a fundamental but crucial task on unsupervised pattern recognition. A recent novel deep dictionary learning system where hierarchical discriminative dictionary learning layers are embedded within a neural network showed either competitive or state-of-the-art results for image classification. We now propose a new method for image clustering, extending hierarchical deep dictionary learning to the unsupervised learning setting. Clustering is induced in the neural network in a simple way by requesting vector outputs associated to categories to approximately lie in the corners of a standard simplex and by approximating a given expected value on such vectors. This is carried out in a scalable way due to the batch-wise updates as clustering is learned in the deep system. We evaluate our proposal on four benchmark datasets and we either achieve competitive results or outperform state-of-the-art clustering methods. Under the unbalanced scenario clustering is known to be more challenging. For this our method additionally considers Bayesian updates to automatically learn the correct cluster proportions for a small number of clusters.

Intra- and Inter-Class Induced Discriminative Deep Dictionary Learning for Visual Recognition

Deep dictionary learning (DDL) aims to learn dictionaries at different levels and the deepest level representations. However, existing DDL algorithms impose a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{1}$</tex-math></inline-formula> -norm constraint on the deepest level representations, ignoring the constraints on different level representations. Meanwhile, they fail to discover effectively the essential discrimination information. Therefore, the obtained representations are less discriminative, which degrades model performance. To tackle those issues, we propose an intra- and inter-class induced discriminative deep dictionary learning (DDDL). Specifically, both intra-class compactness and inter-class separability of layer-wise data representations are newly devised as two discriminative constraints on deep dictionary learning. In a hierarchical structure, we obtain a more informative dictionary and the class-specific representations are thus more discriminative at each layer. Due to the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$l_{2}$</tex-math></inline-formula> -norm intra- and inter-class constraints of layer-wise data representation, we devise a layer-wise optimization strategy to efficiently learn the closed-form solution of the deepest representation for classification. Comprehensive experiments and analyses on several visual recognition tasks show that our DDDL model surpasses recent shallow and deep representation learning approaches.

Discriminative Fisher Embedding Dictionary Transfer Learning for Object Recognition.

In transfer learning model, the source domain samples and target domain samples usually share the same class labels but have different distributions. In general, the existing transfer learning algorithms ignore the interclass differences and intraclass similarities across domains. To address these problems, this article proposes a transfer learning algorithm based on discriminative Fisher embedding and adaptive maximum mean discrepancy (AMMD) constraints, called discriminative Fisher embedding dictionary transfer learning (DFEDTL). First, combining the label information of source domain and part of target domain, we construct the discriminative Fisher embedding model to preserve the interclass differences and intraclass similarities of training samples in transfer learning. Second, an AMMD model is constructed using atoms and profiles, which can adaptively minimize the distribution differences between source domain and target domain. The proposed method has three advantages: 1) using the Fisher criterion, we construct the discriminative Fisher embedding model between source domain samples and target domain samples, which encourages the samples from the same class to have similar coding coefficients; 2) instead of using the training samples to design the maximum mean discrepancy (MMD), we construct the AMMD model based on the relationship between the dictionary atoms and profiles; thus, the source domain samples can be adaptive to the target domain samples; and 3) the dictionary learning is based on the combination of source and target samples which can avoid the classification error caused by the difference among samples and reduce the tedious and expensive data annotation. A large number of experiments on five public image classification datasets show that the proposed method obtains better classification performance than some state-of-the-art dictionary and transfer learning methods. The code has been available at https://github.com/shilinrui/DFEDTL.

DL-Net: Sparsity Prior Learning for Grasp Pattern Recognition

The purpose of grasp pattern recognition is to determine the grasp type for an object to be grasped, which can be applied to prosthetic hand control and ease the burden of amputees. To enhance the performance of grasp pattern recognition, we propose a network DL-Net inspired by dictionary learning. Our method includes two parts: 1) forward propagation for sparsity representation learning and 2) backward propagation for dictionary learning, which utilizes the sparsity prior effectively and learns a discriminative dictionary with stronger expressive ability from a mass of training data. The experiment was performed on two household object datasets, the RGB-D Object dataset, and the Hit-GPRec dataset. The experimental results illustrate that the DL-Net performs better than traditional deep learning methods in grasp pattern recognition.

Towards the characterization of the tumor microenvironment through dictionary learning-based interpretable classification of multiplexed immunofluorescence images

Objective. Histology image analysis is a crucial diagnostic step in staging and treatment planning, especially for cancerous lesions. With the increasing adoption of computational methods for image analysis, significant strides are being made to improve the performance metrics of image segmentation and classification frameworks. However, many developed frameworks effectively function as black boxes, granting minimal context to the decision-making process. Thus, there is a need to develop methods that offer reasonable discriminatory power and a biologically-informed intuition to the decision-making process. Approach. In this study, we utilized and modified a discriminative feature-based dictionary learning (DFDL) paradigm to generate a classification framework that allows for discrimination between two distinct clinical histologies. This framework allows us (i) to discriminate between 2 clinically distinct diseases or histologies and (ii) provides interpretable group-specific representative dictionary image patches, or ‘atoms’, generated during classifier training. This implementation is performed on multiplexed immunofluorescence images from two separate patient cohorts- a pancreatic cohort consisting of cancerous and non-cancerous tissues and a metastatic non-small cell lung cancer (mNSCLC) cohort of responders and non-responders to an immunotherapeutic treatment regimen. The analysis was done at both the image-level and subject-level. Five cell types were selected, namely, epithelial cells, cytotoxic lymphocytes, antigen presenting cells, HelperT cells, and T-regulatory cells, as our phenotypes of interest. Results. We showed that DFDL had significant discriminant capabilities for both the pancreatic pathologies cohort (subject-level AUC-0.8878) and the mNSCLC immunotherapy response cohort (subject-level AUC-0.7221). The secondary analysis also showed that more than 50% of the obtained dictionary atoms from the classifier contained biologically relevant information. Significance. Our method shows that the generated dictionary features can help distinguish patients presenting two different histologies with strong sensitivity and specificity metrics. These features allow for an additional layer of model interpretability, a highly desirable element in clinical applications for identifying novel biological phenomena.

MADPL-net: Multi-layer attention dictionary pair learning network for image classification

With the great success of deep neural networks, combining deep learning with traditional dictionary learning has become a hot issue. However, the performance of these methods is still limited for several reasons. First, some existing methods update dictionary learning and classifier as two independent modules, which limits the classification performance. Second, the non-attention dictionary is learned to represent all images, reducing the model representation flexibility. In this paper, we design a novel end-to-end model named Multi-layer Attention Dictionary Pair Learning Network (MADPL-net), which integrates the learning schemes of the convolutional neural network, deep encoder learning, and attention dictionary pair learning (ADicL) into a unified framework. The encoder layer contains the ADicL block, which selects more image-attentive atoms in the dictionary pair block via the softmax function to ensure MADPL-net classification capability. In addition, ADicL schema can yield discriminative dictionary atoms and feature maps with high inter-class separation and high intra-class compactness. To improve the sparse representation learning performance, MADPL-net adds l1−norm constraint of the analysis dictionary to the cross-entropy loss function. Extensive experiments show that MADPL-net can achieve excellent performance over other state-of-the-arts.

A Bayesian Approach for Joint Discriminative Dictionary and Classifier Learning

Sparse representation has been widely applied to image classification, where the key issue is to extract a suitable discriminative dictionary. To this end, we propose a joint dictionary and classifier learning algorithm based on a parameterized Bayesian model. Therein, the Gaussian priors of a dictionary endow it with the capability of discrimination and representation. Moreover, we introduce a multivariate Gaussian prior for the sparse codes to achieve group sparsity, thereby substantially improving the classification performance. Furthermore, the sparse codes are estimated by a group-sparse Bayesian learning (GSBL) method, and the dictionary atoms are updated sequentially by maximizing a posterior. Moreover, to avoid manual parameter adjustment, the hyperparameters are optimized by an evidence maximization method. Accordingly, we develop a classification scheme via GSBL. Finally, extensive experiments are conducted on six benchmark datasets of face classification, object recognition, handwritten recognition, and scene categorization to substantiate the effectiveness and superiority of the proposed method.

Industrial load identification based on sparse representation classification

In this paper, sparse representation classification (SRC) model based on discriminant dictionary is proposed to train loads of different categories by K-SVD algorithm. Firstly, The absolute value of sparse coefficient and the category of atoms are used to determine the load category. Secondly, the identification of load types is improved in the traditional SRC method based on dictionary learning. Thirdly, the actual identify effect is optimized. Finally, an example is given to illustrate the recognition process. The discriminant dictionary-based SRC method is compared with the traditional SRC method. It proved the feasibility and effectiveness of the method.

Multi-Source Domain Transfer Discriminative Dictionary Learning Modeling for Electroencephalogram-Based Emotion Recognition

Cognitive computing is dedicated to researching a computing principle and method that can simulate the intelligence ability of human brain. Human emotion is the basic component of human cognitive activities. Electroencephalogram (EEG) computer signals obtained from a brain computer interface are difficult to conceal, and using machine learning methods to analyze EEG emotion is a hot topic in artificial intelligence. However, the EEG signal is non-stationary, making it difficult to select sufficient data from the same person to train a classifier for a subject. To promote the performance of emotion recognition methods, a multi-source domain transfer discriminative dictionary learning modeling (MDTDDL) is proposed in this study. The method integrates transfer learning and dictionary learning in a learning model, including the concepts of subspace learning, manifold smoothness, margin-based discriminant embedding, and large margin. The domain-specific transformation matrix projects EEG signals from various domains into the transfer subspace. The domain-invariant dictionary can find potential connections between multiple source domains and target domain. The manifold smoothness and margin-based discriminant embedding term further improve the model’s learning ability. The alternating optimization technique is used in model solving to efficiently compute model parameters. Experiments on the SEED and DEAP datasets demonstrate the effectiveness of MDTDDL.

Representative‐discriminativedictionary learning algorithm for human action recognition using smartphone sensors

SUMMARYWith the advancement of mobile computing, understanding, and interpretation of human activities has become increasingly popular as an innovative human computer interaction application over the past few decades. This article presents a new scheme for action recognition based on sparse representation theory using a novel dictionary learning algorithm. This system employs two types of inertial signals from smartphones namely, accelerometer and gyroscope sensory data. Attainment of higher values of classification accuracy depends on the creation of effective dictionaries that completely retain the important features of every action while maintaining the least correlation with the features of other actions. Accordingly, in this research, we propose a new algorithm for learning dictionaries with two levels of dictionary training that aims at learning a compact, representative, and discriminative dictionary for each class. Unlike typical dictionary learning algorithms that aim at the creation of dictionaries that best represents the features of each class, our proposed algorithm incorporates a discriminative criterion that eventually produces better classification results. To validate the proposed framework, all the experiments were performed using three publicly available datasets.

Support vector machine embedding discriminative dictionary pair learning for pattern classification

Discriminative dictionary learning (DDL) aims to address pattern classification problems via learning dictionaries from training samples. Dictionary pair learning (DPL) based DDL has shown superiority as compared with most existing algorithms which only learn synthesis dictionaries or analysis dictionaries. However, in the original DPL algorithm, the discrimination capability is only promoted via the reconstruction error and the structures of the learned dictionaries, while the discrimination of coding coefficients is not considered in the process of dictionary learning. To address this issue, we propose a new DDL algorithm by introducing an additional discriminative term associated with coding coefficients. Specifically, a support vector machine (SVM) based term is employed to enhance the discrimination of coding coefficients. In this model, a structured dictionary pair and SVM classifiers are jointly learned, and an optimization method is developed to address the formulated optimization problem. A classification scheme based on both the reconstruction error and SVMs is also proposed. Simulation results on several widely used databases demonstrate that the proposed method can achieve competitive performance as compared with some state-of-the-art DDL algorithms.

Discriminative Convolutional Sparse Coding of ECG Signals for Automated Recognition of Cardiac Arrhythmias

Electrocardiogram (ECG) is a common and powerful tool for studying heart function and diagnosing several abnormal arrhythmias. In this paper, we present a novel classification model that combines the discriminative convolutional sparse coding (DCSC) framework with the linear support vector machine (LSVM) classification strategy. In the training phase, most existing convolutional sparse coding frameworks are unsupervised in the sense that label information is ignored in the convolutional filter training stage. In this work, we explicitly incorporate a label consistency constraint called “discriminative sparse-code error” into the objective function to learn discriminative dictionary filters for sparse coding. The learned dictionary filters encourage signals from the same class to have similar sparse codes, and signals from different classes to have dissimilar sparse codes. To reduce the computational complexity, we propose to perform a max-pooling operation on the sparse coefficients. Using LSVM as a classifier, we examine the performance of the proposed classification system on the MIT-BIH arrhythmia database in accordance with the AAMI EC57 standard. The experimental results show that the proposed DCSC + LSVM algorithm can obtain 99.32% classification accuracy for cardiac arrhythmia recognition.

Transfer Discriminative Dictionary Pair Learning Approach for Across-Subject EEG Emotion Classification.

Electroencephalogram (EEG) signals are not easily camouflaged, portable, and noninvasive. It is widely used in emotion recognition. However, due to the existence of individual differences, there will be certain differences in the data distribution of EEG signals in the same emotional state of different subjects. To obtain a model that performs well in classifying new subjects, traditional emotion recognition approaches need to collect a large number of labeled data of new subjects, which is often unrealistic. In this study, a transfer discriminative dictionary pair learning (TDDPL) approach is proposed for across-subject EEG emotion classification. The TDDPL approach projects data from different subjects into the domain-invariant subspace, and builds a transfer dictionary pair learning based on the maximum mean discrepancy (MMD) strategy. In the subspace, TDDPL learns shared synthesis and analysis dictionaries to build a bridge of discriminative knowledge from source domain (SD) to target domain (TD). By minimizing the reconstruction error and the inter-class separation term for each sub-dictionary, the learned synthesis dictionary is discriminative and the learned low-rank coding is sparse. Finally, a discriminative classifier in the TD is constructed on the classifier parameter, analysis dictionary and projection matrix, without the calculation of coding coefficients. The effectiveness of the TDDPL approach is verified on SEED and SEED IV datasets.

Online multitarget tracking system for autonomous vehicles using discriminative dictionary learning with embedded auto‐encoder algorithm

AbstractWith the advancements in 5G network and mobile edge computing technology, autonomous vehicle technology has gained new development opportunities. Multitarget tracking becomes the research hotspots in autonomous vehicles. Since many factors such as motion blur, partial occlusion, and illumination changes affect the performance of target tracking, the problem of target tracking is still an open topic. In this article, inspired by the strong discriminative ability of dictionary learning, the discriminative dictionary learning with embedded auto‐encoder (DDLEA) algorithm is developed for the multitarget tracking system. The DDLEA algorithm integrates the auto‐encoder into the dictionary learning framework and learns sparse representations while preserving the local structure and discriminative information of data. The learned dictionary model has the strong recognition ability. Further, a multitarget tracking system is developed based on the proposed DDLEA algorithm and the hierarchical data association scheme. Based on the target confidence in the STKSVD model, the hierarchical data association method first uses the Hungarian algorithm to complete the preliminary matching of high confidence targets, and then further tracks the low confidence targets to improve the tracking ability. Experiments are carried out the public MOT 2015 dataset. Compared with several popular multitarget tracking algorithms, the tracking performance of our system is satisfactory.

Virtual Kernel Discriminative Dictionary Learning With Weighted KNN for Video Analysis

Recently Kernel-Based Discriminative Dictionary (KDDL) for Video Semantic Content Analysis (VSCA) has become very popular research area, particularly in Human Computer Interactions and Computer Vision decades. Nonetheless, the existing KDDL approaches based on reconstruction error classification, coupled with sparse coefficients do not fully consider discrimination, which is essential for classification performance between video samples, despite their numerous successes. In addition, the size of video samples, an important parameter in kernel-based approaches is mostly ignored. To further improve the accuracy of video semantic classification, a VSC classification approach based on Sparse Coefficient Vector and a Virtual Kernel-based Weighted KNN is proposed in this paper. In the proposed approach, a loss function that integrates reconstruction error and discrimination is put forward. The experimental results show that this method effectively improves recognition and classification accuracy for VSCA compared with some state-of-the-art baseline approaches.