Discriminative Learning Algorithm Research Articles

A novel cost-sensitive three-way intuitionistic fuzzy large margin classifier

Three-way decision (3WD) has been widely applied in diverse fields in tackling uncertainty, particularly in classification domain. As a discriminative learning algorithm, Large Margin Distribution Machine (LDM) aims to maximize the inter-class margin by leveraging the marginal distribution of samples, which captures the underlying structure and achieves superior classification performance. However, existing LDMs still suffer from some inherent flaws, including: i) the neglect of fuzzy decision items, leading to the inadequate handling of uncertainty; ii) the utilization of a cost-insensitive mechanism, resulting in high misclassification costs; and iii) the disregard of sample credibility, contributing to the noise susceptibility. To address these limitations, we introduce the intuitionistic fuzzy (IF) and cost-sensitive 3WD (CS3WD), presenting an innovative model known as the CS3W-IFLMC model. The IF theory is employed to quantify sample confidence levels, augmenting noise suppression in our proposed model. Moreover, the integration of the CS3WD method effectively reduces the overall decision cost and further improves the handling of uncertainty in datasets. Consequently, the CS3W-IFLMC model demonstrates superior noise resilience and generalization performance. Our comparative experiments validate the efficacy of the proposed CS3W-IFLMC model in achieving robustness to noise while upholding a competitive classification performance.

Cross-dataset discriminant subspace learning algorithm for apple leaf diseases identification

Apple has a large planting area and high yield in China, but it is easily affected by diseases. Artificial intelligence technology has achieved good results in apple leaf disease identification. However, the training and testing data in complex environments often come from different collection scenarios. There are significant differences in the training and testing datasets, which leads to a significant decline in the identification performance. To improve the generalization ability of apple leaf disease identification, we develop a cross-dataset discriminant subspace learning (CDDSL) algorithm by utilizing the idea of transfer learning, low-rank sparse representation, and maximum margin neighborhood preserving embedding (NPE) to cross-dataset scenarios. Firstly, based on the transfer learning and low-rank sparse representation, each target sample can be represented by a linear combination of source domain samples. The sparse constraint on the noise matrix weakens the influence of noise in the sample data. Then, CDDSL progressively projects the subspace features into the semantic space and establishes a discriminant classifier. Meanwhile, CDDSL utilizes the maximal margin NPE to improve the intraclass compactness and interclass separability of projection features, which can enhance the image identification performance. The experiments conducted on real-world apple leaf datasets verify the effectiveness of the CDDSL algorithm in cross-dataset scenario.

Stacking multi-view broad learning system with residual structures for classification

Broad learning system (BLS) is an effective and efficient discriminative learning algorithm, particularly adept at rapid implementation for incremental learning without necessitating substantial computational resources. However, BLS exhibits excellent performance exclusively in low-complexity scenarios, with its classification performance on RGB images being somewhat underwhelming. In this article, a novel BLS model named the stacking multi-view broad learning system with residual structures (RSM-BLS) is proposed. This model integrates the strengths of residual structures, multi-view learning, and transfer learning. Furthermore, the specific architecture of this model and the process of implementing incremental learning are provided. Finally, we evaluate the classification performance of the proposed RSM-BLS on the NOBR dataset, Fashion-MNIST dataset, cifar10 dataset, SVHN dataset, and cifar100 dataset, and conduct ablation studies and incremental structure tests. The experimental results indicate that, in comparison to relevant BLS algorithms and state-of-the-art methods, RSM-BLS exhibits superior performance and generalization capabilities.

Robust discriminant latent variable manifold learning for rotating machinery fault diagnosis

Fault diagnosis is an important technology for performing intelligent manufacturing. To simultaneously maintain high manufacturing quality and low failure rate for manufacturing systems, it is of great value to accurately locate the fault element, evaluate the fault severity and find the fault root cause. In order to effectively perform feature extraction for rotating machinery fault diagnosis, a robust discriminant latent variable manifold learning (RDLVML) algorithm is proposed in this paper. Specifically, RDLVML algorithm could effectively select the features of the high-dimensional fault data, and extract the low-dimensional fault features with better discrimination. Through the idea of discriminative learning, the accuracy of fault diagnosis was improved by increasing the aggregation of the same state feature samples and the difference between different classes of feature samples. A novel weighted neighborhood graph is proposed by constructing the q-Rényi kernel function, which could effectively suppress the interference of outliers and noise, and make the RDLVML algorithm more robust. Moreover, the fault diagnosis method of rotating machinery based on RDLVML is proposed in this paper. Through experimental verifications and comparisons with several classical feature selection algorithms, rotating machinery fault diagnosis could be more accurately performed by this method.

Discriminative estimation of probabilistic context-free grammars for mathematical expression recognition and retrieval

We present a discriminative learning algorithm for the probabilistic estimation of two-dimensional probabilistic context-free grammars (2D-PCFG) for mathematical expressions recognition and retrieval. This algorithm is based on a generalization of the H-criterion as the objective function and the growth transformations as the optimization method. For the development of the discriminative estimation algorithm, the N-best interpretations provided by the 2D-PCFG have been considered. Experimental results are reported on two available datasets: Im2Latex and IBEM. The first experiment compares the proposed discriminative estimation method with the classic Viterbi-based estimation method. The second one studies the performance of the estimated models depending on the length of the mathematical expressions and the number of admissible errors in the metric used.

Deep Generative Models in the Industrial Internet of Things: A Survey

Advances in communication technologies and artificial intelligence are accelerating the paradigm of industrial Internet of Things (IIoT). With IIoT enabling continuous integration of sensors and controllers with the network, intelligent analysis of the generated Big Data is a critical requirement. Although IIoT is considered a subset of IoT, it has its own peculiarities in terms of higher levels of safety, security, and low-latency communication in an environment of critical real-time operations. Under these circumstances, discriminative deep learning (DL) algorithms are unsuitable due to their need for large amounts of labeled and balanced training data, uncertainty of inputs, etc. To overcome these issues, researchers have started using deep generative models (DGMs), which combine the flexibility of DL with the inference power of probabilistic modeling. In this article, we review the state of the art of DGMs and their applicability to IIoT, classifying the reviewed works into the IIoT application areas of anomaly detection, trust-boundary protection, network traffic prediction, and platform monitoring. Following an analysis of existing IIoT DGM implementations, we identify challenges (i.e., weak discriminative capability, insufficient interpretability, lack of generalization ability, generated data vulnerability, privacy concern, and data complexity) that need to be investigated in order to accelerate the adoption of DGMs in IIoT and also propose some potential research directions.

Discriminative aging subspace learning for age estimation

Human age estimation from facial images has become an active research topic in computer vision field because of various real-world applications. Temporal property of facial aging display sequential patterns that lie on the low-dimensional aging manifold. In this paper, we propose hidden factor analysis (HFA) model-based discriminative manifold learning method for age estimation. The hidden factor analysis decomposes facial features into independent age factor and identity factor. Various age invariant face recognition systems in the literature utilize identity factor for face recognition; however, the age factor remains unutilized. The age component of the hidden factor analysis model depends on the subject’s age. Thus it carries significant age-related information. In this paper, we demonstrate that such aging patterns can be effectively extracted from the HFA-based discriminant subspace learning algorithm. Next, we have applied multiple regression methods on low-dimensional aging features learned from the HFA model. Effect of reduced dimensionality on the accuracy has been evaluated by extensive experiments and compared with the state-of-the-art methods. Effectiveness and robustness in terms of MAE and CS of the proposed framework are demonstrated using experimental analysis on a large-scale aging database MORPH II. The accuracy of our method is found superior to the current state-of-the-art methods.

Fuzzy Discriminative Block Representation Learning for Image Feature Extraction.

Representation learning is widely used to project high-dimensional data to low-dimensional subspace for feature extraction in image recognition tasks. However, many related methods barely explore the fuzziness and uncertainty between data classes. Besides, the classical unsupervised sparse constraint weakens the evaluation of feature importance and neglects the preservation of discriminant information during sparse representation. To solve these issues, a novel fuzzy discriminative block representation learning (FDBRL) algorithm is proposed for image feature extraction. FDBRL aims to enhance the discriminability of subspace by designing effective constraints for projection learning. Specifically, based on the label information and the fuzzy relation between data, we construct a fuzzy block weight matrix and embed it into the l2,1 norm regularization term to realize supervised sparse constraint for the representation learning. Next, the low-rank constraint is used to capture the inherent global structure information of data. Finally, we introduce a classification loss term with transformation matrix for joint optimization, such that the projection learning is not limited to number of classes, and the discriminative ability is further improved. Comprehensive experimental results on six benchmarks verify that our method achieves promising performance with other state-of-the-arts in both robustness and effectiveness.

Discriminative shared transform learning for sketch to image matching

Sketch to digital image matching refers to the problem of matching a sketch image (often drawn by hand or created by a software) against a gallery of digital images (captured via an acquisition device such as a digital camera). Automated sketch to digital image matching has applicability in several day to day tasks such as similar object image retrieval, forensic sketch matching in law enforcement scenarios, or profile linking using caricature face images on social media. As opposed to the digital images, sketch images are generally edge-drawings containing limited (or no) textural or colour based information. Further, there is no single technique for sketch generation, which often results in varying artistic or software styles, along with the interpretation bias of the individual creating the sketch. Beyond the variations observed across the two domains (sketch and digital image), automated sketch to digital image matching is further marred by the challenge of limited training data and wide intra-class variability. In order to address the above problems, this research proposes a novel Discriminative Shared Transform Learning (DSTL) algorithm for sketch to digital image matching. DSTL learns a shared transform for data belonging to the two domains, while modeling the class variations, resulting in discriminative feature learning. Two models have been presented under the proposed DSTL algorithm: (i) Contractive Model (C-Model) and (ii) Divergent Model (D-Model), which have been formulated with different supervision constraints. Experimental analysis on seven datasets for three case studies of sketch to digital image matching demonstrate the efficacy of the proposed approach, highlighting the importance of each component, its input-agnostic behavior, and improved matching performance.

Discriminative learning of bayesian network parameters by differential evolution

This work proposes Differential Evolution (DE) to train parameters of Bayesian Networks (BN) for optimizing the Conditional Log-Likelihood (Discriminative Learning) instead of the log-likelihood (Generative Learning). Any given BN structure encodes assumptions about conditional independencies among the attributes and will result in error if they do not hold in the data. Such an error includes the classification dimension. The main goal of Discriminative learning is minimize this type of error. In this sense, although BN Discriminative Parameter Learning algorithms have been proposed, to the best of the authors’ knowledge, a meta-heuristic approach has not been devised yet. Thus, this is our main contribution: to come up with this kind of solution and evaluate its behavior so that its feasibility in this domain can be determined. Regarding the proposed method, the bias provided by the best solution in the population improves DE’s performance. DE variants based on JADE, such as L-SHADE and C-JADE, are especially recommended when introducing adaptation mechanisms of mutation and crossover parameters thus reducing the dependence on their calibration. L-SHADE is computationally recommended over any other DE variant. DE approach works well in essentially every standard situation, so DE approach is robust and at least as good, and often better, than the state-of-art method for Discriminative Learning called WANBIA. Our results suggest a potential benefit for Discriminative parameter learning with strong independence assumptions among attributes and that it typically produces more accurate classifiers than generative learning.

Image-driven discriminative and generative machine learning algorithms for establishing microstructure–processing relationships

We investigate the methods of microstructure representation for the purpose of predicting processing condition from microstructure image data. A binary alloy (uranium–molybdenum) that is currently under development as a nuclear fuel was studied for the purpose of developing an improved machine learning approach to image recognition, characterization, and building predictive capabilities linking microstructure to processing conditions. Here, we test different microstructure representations and evaluate model performance based on the F1 score. A F1 score of 95.1% was achieved for distinguishing between micrographs corresponding to ten different thermo-mechanical material processing conditions. We find that our newly developed microstructure representation describes image data well, and the traditional approach of utilizing area fractions of different phases is insufficient for distinguishing between multiple classes using a relatively small, imbalanced original dataset of 272 images. To explore the applicability of generative methods for supplementing such limited datasets, generative adversarial networks were trained to generate artificial microstructure images. Two different generative networks were trained and tested to assess performance. Challenges and best practices associated with applying machine learning to limited microstructure image datasets are also discussed. Our work has implications for quantitative microstructure analysis and development of microstructure–processing relationships in limited datasets typical of metallurgical process design studies.

A novel intelligent detection schema of series arc fault in photovoltaic (PV) system based convolutional neural network

Series Arc Fault (SAF) can be defined as the failure that occurs between any electrical contact and any electrical circuitry. However, it considered one of the common failures that affect the operation of the PV system and causes serious problems such as fires and electrical shock hazards. Several reasons increase the possibility of this type of failures such as incorrect installation, irregular maintenance, and some environmental effects. This paper presents a new intelligent and accurate detection method of SAF in the PV system. In this method, Convolutional neural networks (CNN) which is a discriminative (supervised) deep learning algorithm used for the process of fault detection. In normal cases, the signal consists of DC component, inverter component and noise of Network. In the case of SAF, a new component will add to the signal; therefore, CNN used to discriminate against the new component to accurately detect the SAF. PSCAD is used to generate the Arc fault model; Performance evaluation and the results of the proposed method implemented using Python. The achieved accuracy of the proposed detection method is 98.9%. .

Discriminative training of feed-forward and recurrent sum-product networks by extended Baum-Welch

We present a discriminative learning algorithm for feed-forward Sum-Product Networks (SPNs) [42] and recurrent SPNs [31] based on the Extended Baum-Welch (EBW) algorithm [4]. We formulate the conditional data likelihood in the SPN framework as a rational function, and we use EBW to monotonically maximize it. We derive the algorithm for SPNs and RSPNs with both discrete and continuous variables. The experiments show that this algorithm performs better than both generative Expectation-Maximization, and discriminative gradient descent on a wide variety of applications. We also demonstrate the robustness of the algorithm in the case of missing features by comparing its performance to Support Vector Machines and Neural Networks.

Discriminant sub-dictionary learning with adaptive multiscale superpixel representation for hyperspectral image classification

Sparse representation and dictionary learning have been successfully applied in hyperspectral image classification. Generally, it is more effective to learn the sub-dictionary for each class and utilize multiple scale strategy. However, the sub-dictionary may only consider the within-class information and ignore the discriminative information. For multiscale sparse representation, the shape of the regions would not adaptively change according to context structure. This paper proposes a discriminative sub-dictionary learning algorithm and an adaptive multiscale superpixel classification strategy under sparse representation framework for hyperspectral image classification. In dictionary learning stage, the global constraint term can ensure the learnt sub-dictionaries contain more discriminative information. In sparse representation stage, the adaptive multiscale superpixel strategy can exploit the spatial context according to the structural information. The combination of the two strategies can further explore the spatial and spectral information in hyperspectral image. Experiments on four hyperspectral image datasets prove that the proposed method has a significant improvement in both quantitative and qualitative analysis comparing with state-of-the-art algorithms. Moreover, the proposed algorithm can produce satisfactory results for the imbalanced data problem involved in hyperspectral image.

Multi-empirical Discriminant Multi-Agent Reinforcement Learning Algorithm Based on Intra-group Evolution

In order to find the optimal target in the unknown environment more quickly, a multi-empirical discriminant multi-agent reinforcement learning algorithm based on intra-group evolution is proposed based on the deep deterministic policy gradient algorithm (DDPG). In the unknown environment, the agent can find the target faster by performing Information exchange, group genetics and other mechanisms. The comparison with the traditional reinforcement learning algorithm shows that the proposed algorithm is superior to the traditional reinforcement learning algorithm in solving time and solving accuracy, and can quickly and effectively find the optimal target in the environment.CCS Concepts•Computing methodologies → Multi-agent systems

Multichannel DNA Sensor Array Fingerprints Cell States and Identifies Pharmacological Effectors of Catabolic Processes.

Cells at disease onset are often associated with subtle changes in the expression level of a single or few molecular components, making traditionally used biomarker-driven clinical diagnosis a challenging task. We demonstrate here the design of a DNA nanosensor array with multichannel output that identifies the normal or pathological state of a cell based on the alteration of its global proteomic signature. Fluorophore-encoded single-stranded DNA (ssDNA) strands were coupled via supramolecular interaction with a surface-functionalized gold nanoparticle quencher to generate this integrated sensor array. In this design, ssDNA sequences exhibit dual roles, where they provide differential affinities with the receptor gold nanoparticle as well as act as transducer elements. The unique interaction mode of the analyte molecules disrupts the noncovalent supramolecular complexation, generating simultaneous multichannel fluorescence output to enable signature-based analyte identification via a linear discriminant analysis-based machine learning algorithm. Different cell types, particularly normal and cancerous cells, were effectively distinguished using their fluorescent fingerprints. Additionally, this DNA sensor array displayed excellent sensitivity to identify cellular alterations associated with chemical modulation of catabolic processes. Importantly, pharmacological effectors, which could modulate autophagic flux, have been effectively distinguished by generating responses from their global protein signatures. Taken together, these studies demonstrate that our multichannel DNA nanosensor is well suited for rapid identification of subtle changes in a complex mixture and thus can be readily expanded for point-of-care clinical diagnosis, high-throughput drug screening, or predicting the therapeutic outcome from a limited sample volume.

Discriminative Fast Hierarchical Learning for Multiclass Image Classification.

In this article, a discriminative fast hierarchical learning algorithm is developed for supporting multiclass image classification, where a visual tree is seamlessly integrated with multitask learning to achieve fast training of the tree classifier hierarchically (i.e., a set of structural node classifiers over the visual tree). By partitioning a large number of categories hierarchically in a coarse-to-fine fashion, a visual tree is first constructed and further used to handle data imbalance and identify the interrelated learning tasks automatically (e.g., the tasks for learning the node classifiers for the sibling child nodes under the same parent node are strongly interrelated), and a multitask SVM classifier is trained for each nonleaf node to achieve more effective separation of its sibling child nodes at the next level of the visual tree. Both the internode visual similarities and the interlevel visual correlations are utilized to train more discriminative multitask SVM classifiers and control the interlevel error propagation effectively, and a stochastic gradient descent (SGD) algorithm is developed for learning such multitask SVM classifiers with higher efficiency. Our experimental results have demonstrated that our fast hierarchical learning algorithm can achieve very competitive results on both the classification accuracy rates and the computational efficiency.

Robust Discriminative Metric Learning for Image Representation

Metric learning has attracted significant attention in the past decades, because of its appealing advances in various real-world tasks, e.g., person re-identification and face recognition. Traditional supervised metric learning attempts to seek a discriminative metric, which could minimize the pairwise distance of within-class data samples, while maximizing the pairwise distance of data samples from various classes. However, it is still a challenge to build a robust and discriminative metric, especially for corrupted data in the real-world application. In this paper, we propose a Robust Discriminative Metric Learning algorithm through fast low-rank representation and denoising strategy. To be specific, the metric learning problem is guided by a discriminative regularization by incorporating the pair-wise or class-wise information. Moreover, the low-rank basis learning is jointly optimized with the metric to better uncover the global data structure and remove noise. Furthermore, the fast low-rank representation is implemented to mitigate the computational burden and ensure the scalability on large-scale datasets. Finally, we evaluate our learned metric on several challenging tasks, e.g., face recognition/verification, object recognition, image clustering, and person re-identification. The experimental results verify the effectiveness of our proposed algorithm in comparison to many metric learning algorithms, even deep learning ones.

Using Hybrid Discriminative-Generative Models for Binary Classification

Discriminative and generative machine learning algorithms have been successfully used in different classification tasks during the last several decades. They both have some advantages and disadvantages and depending on a problem, one type of algorithm performs better than the other one. In this paper we contribute to the research of combination of both approaches and propose literature based a hybrid discriminative-generative generic model. Also, we propose hybrid model structure finding and building a new algorithm. We present theoretical and practical advantages of the hybrid model over its consisting algorithms, efficiency of the model structure finding algorithm, then perform experiments and compare results.

Hyperspectral Image Classification Method Based on CNN Architecture Embedding With Hashing Semantic Feature

Deep convolutional neural networks (CNN) have led to a successful breakthrough for hyperspectral image (HSI) classification. In this paper, a CNN system embedded with an extracted hashing feature is proposed for HSI classification that utilizes the semantic information of the HSI. First, a series of hash functions are constructed to enhance the presentation of the locality and discriminability of classes. Then, the sparse binary hash codes calculated by the discriminative learning algorithm are combined into the original HSI. Next, we design a CNN framework with seven hidden layers to obtain the hierarchical feature maps with both spectral and spatial information for classification. A deconvolution layer aims to improve the robustness of the proposed CNN network and is used to enhance the expression of deep features. The proposed CNN classification architecture achieves powerful distinguishing ability from different classes. The extensive experiments on real hyperspectral images results demonstrate that the proposed CNN network can effectively improve the classification accuracy after the embedding of the extracted semantic features.