Discriminative Extreme Learning Machine Research Articles

Multilayer discriminative extreme learning machine for classification

Multilayer discriminative extreme learning machine for classification

Discriminative Extreme Learning Machine with Cross-Domain Mean Approximation for Unsupervised Domain Adaptation

Extreme Learning Machine (ELM) is widely used in various fields because of its fast training and high accuracy. However, it does not primarily work well for Domain Adaptation (DA) in which there are many annotated data from auxiliary domain and few even no annotated data in target domain. In this paper, we propose a new variant of ELM called Discriminative Extreme Learning Machine with Cross-Domain Mean Approximation (DELM-CDMA) for unsupervised domain adaptation. It introduces Cross-Domain Mean Approximation (CDMA) into the hidden layer of ELM to reduce distribution discrepancy between domains for domain bias elimination, which is conducive to train a high accuracy ELM on annotated data from auxiliary domains for target tasks. Linear Discriminative Analysis (LDA) is also adopted to improve the discrimination of learned model and obtain higher accuracy. Moreover, we further provide a Discriminative Kernel Extreme Learning Machine with Cross-Domain Mean Approximation (DKELM-CDMA) as the kernelization extension of DELM-CDMA. Some experiments are performed to investigate the proposed approach, and the result shows that DELM-CDMA and DKELM-CDMA could effectively extend ELM suitable for domain adaptation and outperform ELM and many other domain adaptation approaches.

Aircraft Type Recognition in Remote Sensing Images: Bilinear Discriminative Extreme Learning Machine Framework

Small inter-class and massive intra-class changes are important challenges in aircraft model recognition in the field of remote sensing. Although the aircraft model recognition algorithm based on the convolutional neural network (CNN) has excellent recognition performance, it is limited by sample sets and computing resources. To solve the above problems, we propose the bilinear discriminative extreme learning machine (ELM) network (BD-ELMNet), which integrates the advantages of the CNN, autoencoder (AE), and ELM. Specifically, the BD-ELMNet first executes the convolution and pooling operations to form a convolutional ELM (ELMConvNet) to extract shallow features. Furthermore, the manifold regularized ELM-AE (MRELM-AE), which can simultaneously consider the geometrical structure and discriminative information of aircraft data, is developed to extract discriminative features. The bilinear pooling model uses the feature association information for feature fusion to enhance the substantial distinction of features. Compared with the backpropagation (BP) optimization method, BD-ELMNet adopts a layer-by-layer training method without repeated adjustments to effectively learn discriminant features. Experiments involving the application of several methods, including the proposed method, to the MTARSI benchmark demonstrate that the proposed aircraft type recognition method outperforms the state-of-the-art methods.

A Weighted Discriminative Extreme Learning Machine Design for Lung Cancer Detection by an Electronic Nose System

This article presents a study on lung cancer detection based on electronic nose technology. The pattern recognition algorithm is extremely crucial for an electronic nose system, but the customary learning algorithms usually prefer the majority class for class imbalance learning due to the assumption of equal misclassification costs. To address this challenge, in this article, we propose a novel classification method named weighted discriminative extreme learning machine (WDELM) for lung cancer diagnosis. First, the WDELM assigns a different weight to each particular sample by using a flexible weighting strategy, which enables it to perform classification tasks with the unbalanced class distribution. Then, an alternating iterative algorithm is proposed to solve the convex objective function with a theoretical analysis presented. Finally, the effectiveness of the proposed method has been evaluated on lung cancer datasets and public datasets by comparing it with conventional methods. Experimental results have confirmed that the WDELM surpasses conventional methods and is competent for lung cancer diagnosis.

Land-use classification via ensemble dropout information discriminative extreme learning machine based on deep convolution feature

Classifying land-use scenes with high quality and accuracy is an important research direction in current hyperspectral remote sensing images, which is conducive to scientific management and utilization of land. An effective classifier and feature extractor can improve classification stability and accuracy. Therefore, based on deep learning technique, a dropout-based ensemble learning method is proposed in this paper, which combines convolutional neural network (CNN) and information discriminating extreme learning machine (IELM). Pre-trained CNN is used to learn effective and robust features, and deep convolution features are fed to the IELM classifier. Then the adoption of dropout technique and ensemble method can improve generalization capabilities and stability. The effectiveness of the proposed algorithm is tested by hyperspectral remote sensing image classification experiments. The experimental results show that the proposed E-CNN-dropIELM has achieved satisfactory results compared to state-of-the-art methods in terms of classification accuracy and stability.

A Joint Unsupervised Cross-Domain Model via Scalable Discriminative Extreme Learning Machine

Extreme learning machine (ELM) is a well-known cognitive model, that has been extended to cross-domain tasks. Nonetheless, most existing paradigms that are based on ELM either concern about a case in which a specific number of instances are labelled in the target domain or a learner is trained without sufficient capacity to eliminate the gap between domains. To cope with the scenario in which there are no target labels and to acquire a better adaptive learner, we propose a joint unsupervised cross-domain model via scalable discriminative ELM, which is abbreviated as JUC-SDELM. Within the framework, the scalable factor is integrated into discriminative ELM (DELM) to adjust the output margin, which strengthens the discriminative capacity of the ELM classifier. In addition, we follow the basic strategy of joint distribution adaptation (JDA) to align the subspaces generated by JUC-SDELM in terms of their statistics. The discrepancy across domains is alleviated after a few iterations. Moreover, a metric on the outputs of ELM is utilized to filter unreliable pseudo labels in the target domain, with the aim of eliminating the negative transfer effect. Results are obtained by comparing JUC-SDELM with state-of-the-art baseline methods on 16 cross-domain benchmarks that were constructed based on three combined datasets. Likewise, the outcomes in terms of key parameters are also examined. According to the experiments, our proposed model achieves competitive overall performance.

A Novel Hyperspectral Image Clustering Method With Context-Aware Unsupervised Discriminative Extreme Learning Machine

The extension of supervised extreme learning machine (ELM) to unsupervised one, which involves discriminative and manifold regularization, is increasingly gaining attention in hyperspectral image (HSI) clustering. This is due to the fact that HSI clustering problem requires a spectral-spatial feature extraction mechanism that must fully exploit local spectral-spatial contexts and global discriminative information to reduce the misclassification while improve the robustness in clustering procedural. In this paper, we propose a novel context-aware unsupervised discriminative ELM method for HSI clustering. The main novelty of the proposed method are twofold:1) a local spectral-spatial context integration and reshaping mechanism is incorporated into the hidden layer feature representation by using a context-aware propagation filtering procedure; and 2) both local manifold and global discriminative regularization are integrated into unsupervised ELM framework to learn an effective data representation. The most important advantage of the proposed method is that it efficiently exploits the spatial contextual information of HSI through a propagation filtering procedural; furthermore, the learned data representation can capture the intrinsic structure by exploiting the local manifold and global information by discriminative regularization. Experimental results show that the proposed algorithm obtains a competitive performance and outperforms other state of the art ELM-based methods and the other unsupervised methods.

Neuron Pruning-Based Discriminative Extreme Learning Machine for Pattern Classification

Extreme learning machine (ELM), as a newly developed learning paradigm for the generalized single hidden layer feedforward neural networks, has been widely studied due to its unique characteristics, i.e., fast training, good generalization, and universal approximation/classification ability. A novel framework of discriminative extreme learning machine (DELM) is developed for pattern classification. In DELM, the margins between different classes are enlarged as much as possible through a technique called e-dragging. DELM is further extended to pruning DELM (P-DELM) using L2,1-norm regularization. The performance of DELM is compared with several state-of-the-art methods on public face databases. The simulation results show the effectiveness of DELM for face recognition when there are posture, facial expression, and illumination variations. P-DELM can distinguish the importance of different hidden neurons and remove the worthless ones. The model can achieve promising performance with fewer hidden neurons and less prediction time on several benchmark datasets. In DELM model, the margins between different classes are enlarged by learning a nonnegative label relaxation matrix. The experiments validate the effectiveness of DELM. Furthermore, DELM is extended to P-DELM based on L2,1-norm regularization. The developed P-DELM can naturally distinguish the importance of different hidden neurons, which will lead to a more compact network by neuron pruning. Experimental validations on some benchmark datasets show the advantages of the proposed P-DELM method.

Hyperspectral remote sensing image classification with information discriminative extreme learning machine

Hyperspectral remote sensing image classification is important aspect of current research. Extreme learning machine (ELM) has been widely used in the field of pattern recognition for its efficient and good generalization performance. With the study of hyperspectral remote sensing image classification, this paper proposes an information discriminative extreme learning machine (IELM). IELM inherits the advantages of ELM, can solve the problems that ELM learning is insufficient for hyperspectral remote sensing image with limited scale of sample data. The proposed algorithm is tested by experiments for hyperspectral remote sensing image classification. The experiment results show that the proposed algorithm has better classification effect.

Discriminative extreme learning machine with supervised sparsity preserving for image classification

In order to seek non-propagation method to train generalized single-hidden layer feed forward neural networks, extreme learning machine was proposed, which has been proven to be an effective and efficient model for both multi-class classification and regression. Different from most of existing studies which consider extreme learning machine as a classifier, we make improvements on it to let it become a feature extraction model in this paper. Specifically, a discriminative extreme learning machine with supervised sparsity preserving (SPELM) model is proposed. From the hidden layer to output layer, SPELM performs as a subspace learning method by considering the discriminative as well as sparsity information of data. The sparsity information of data is identified by solving a supervised sparse representation objective. Experiments are conducted on four widely used image benchmark data sets and the classification results demonstrate the effectiveness of the proposed SPELM model.

Information discriminative extreme learning machine

Extreme learning machine (ELM) has become one of the new research hotspots in the field of pattern recognition and machine learning. However, the existing extreme learning machine algorithms cannot better use identification information of data. Aiming at solving this problem, we propose a regularized extreme learning machine (algorithm) based on discriminative information (called IELM). In order to evaluate and verify the effectiveness of the proposed method, experiments use widely used image data sets. The comparative experimental results show that the proposed algorithm in the paper can significantly improve the classification performance and generalization ability of ELM.

Robust discriminative extreme learning machine for relevance feedback in image retrieval

Relevance feedback (RF) has long been an important approach for multi-media retrieval because of the semantic gap in image content, where SVM based methods are widely applied to RF of content-based image retrieval. However, RF based on SVM still has some limitations: (1) the high dimension of image features always make the RF time-consuming; (2) the model of SVM is not discriminative, because labels of image features are not sufficiently exploited. To solve above problems, we proposed robust discriminative extreme learning machine (RDELM) in this paper. RDELM involved both robust within-class and between-class scatter matrices to enhance the discrimination capacity of ELM for RF. Furthermore, an angle criterion dimensionality reduction method is utilized to extract the discriminative information for RDELM. Experimental results on four benchmark datasets (Corel-1K, Corel-5K, Corel-10K and MSRC) illustrate that our proposed RF method in this paper achieves better performance than several state-of-the-art methods.

An unsupervised discriminative extreme learning machine and its applications to data clustering

Extreme Learning Machine (ELM), which was initially proposed for training single-layer feed-forward networks (SLFNs), provides us a unified efficient and effective framework for regression and multiclass classification. Though various ELM variants were proposed in recent years, most of them focused on the supervised learning scenario while little effort was made to extend it into unsupervised learning paradigm. Therefore, it is of great significance to put ELM into learning tasks with only unlabeled data. One popular approach for mining knowledge from unlabeled data is based on the manifold assumption, which exploits the geometrical structure of data by assuming that nearby points will also be close to each other in transformation space. However, considering the manifold information only is insufficient for discriminative tasks. In this paper, we propose an improved unsupervised discriminative ELM (UDELM) model, whose main advantage is to combine the local manifold learning with global discriminative learning together. UDELM can be efficiently optimized by solving a generalized eigen-value decomposition problem. Extensive comparisons over several state-of-the-art models on clustering image and emotional EEG data demonstrate the efficacy of UDELM.