Ensemble Of Multiple Classifiers Research Articles

Dual contrastive universal adaptation network for multi-source visual recognition

This paper explores recently proposed Universal Multi-source Domain Adaptation (UniMDA) task. UniMDA task is different from existing domain adaptation (DA) tasks, e.g., Multi-source DA, Closed set DA or Universal DA. UniMDA not only handles multi-source issue, but also needs no prior knowledge about the overlap between the target and source label sets. UniMDA task has three challenges: (i) Domain shift issue among the multiple source domains. (ii) Domain shift issue between target and source domains. (iii) Category shift between the target and each source. Towards tackling the challenges, we formulate a universal multi-source adaptation network termed as Multi-Source Dual Contrastive Network (MSDCN), including a transferability rule and contrastive module. In addition, to handle multi-source scenario, we propose (1) pairwise similarities maximization over the examples from multiple source domains, and (2) alternative optimization strategy for training the ensemble of multiple source classifiers end-to-end. The proposed method can handle UniMDA scenario generally, where the label set in each source domain may be different from the target domain, while maintaining the complexity of the method in different DA scenarios. Experiments are conducted on several real-world multi-source benchmarks. The results show that MSDCN could work stably, and exceed the state-of-the-art performance against existing domain adaptation algorithms.

Multi-Scale Convolutional Neural Network Ensemble for Multi-Class Arrhythmia Classification.

The automated analysis of electrocardiogram (ECG) signals plays a crucial role in the early diagnosis and management of cardiac arrhythmias. The diverse etiology of arrhythmia and the subtle variations in the pathological ECG characteristics pose challenges in designing reliable automated methods. Existing methods mostly use single deep convolutional neural networks (DCNN) based approaches for arrhythmia classification. Such approaches may not be adequate for effectively representing diverse pathological ECG characteristics. This paper presents a novel way of using an ensemble of multiple DCNN classifiers for effective arrhythmia classification named Deep Multi-Scale Convolutional neural network Ensemble (DMSCE). Specifically, we designed multiple scale-dependent DCNN expert classifiers with different receptive fields to encode the scale-specific pathological ECG characteristics and generate the local predictions. A convolutional gating network is designed to compute the dynamic fusion weights for the experts based on their competencies. These weights are used to aggregate the local predictions and generate final diagnosis decisions. Moreover, a new error function with a correlation penalty is formulated to enable interaction and optimal diversity among experts during the training process. The model is evaluated on the PTBXL-2020 12-lead ECG and the CinC-training2017 single-lead ECG datasets and delivers state-of-the-art performance. Average F1-score of 84.5 % and 88.3 % are obtained for the PTBXL-2020 and the CinC-training2017 datasets, respectively. Impressive performance across various cardiac arrhythmias and the elegant generalization ability for different leads make the method suitable for reliable remote or in-hospital arrhythmia monitoring applications.

Multisensor-Based Heavy Machine Faulty Identification Using Sparse Autoencoder-Based Feature Fusion and Deep Belief Network-Based Ensemble Learning

Faulty identification plays a vital role in the area of prognostic and the health management (PHM) of the industrial equipment which offers great support to the maintenance strategy decision. Owing to the complexity of the machine internal component-system structure, the precise prediction of the heavy machine is hard to be obtained, thus full of uncertainty. Moreover, even for a single component, the feature representation of the acquired conditional monitoring signal can be different due to the different deployment of the sensor location and environmental inference, causing difficulty in feature selection and uncertainty in faulty identification. In order to improve the model identification reliability, a novel hybrid machine faulty identification approach based on sparse autoencoder- (SAE-) and deep belief network- (DBN-) based ensemble learning is proposed in this paper. First, six kinds of statistical features are extracted and normalized from multiple sensors monitoring the same target component. Second, the six extracted features are fused by the two-stage SAE proposed in this paper from the sensor dimension and feature dimension, respectively. The composite feature fused in the feature dimension is regarded as the comprehensive representation of the corresponding component. Finally, the fused features containing comprehensive representation of different components are utilized to predict the machine health condition by the ensemble of multiple deep belief classifiers. The effectiveness of the proposed method is validated by the two case studies of wind turbine gearbox and industrial port crane. The experimental result shows that the proposed ensemble learning approach outperforms other traditional deep learning approaches in terms of the prediction accuracy and the prediction stability when dealing with multisensor feature fusion and the precise faulty identification of the industrial heavy machine.

Progress in Grassland Cover Conservation in Southern European Mountains by 2020: A Transboundary Assessment in the Iberian Peninsula with Satellite Observations (2002–2019)

Conservation and policy agendas, such as the European Biodiversity strategy, Aichi biodiversity (target 5) and Common Agriculture Policy (CAP), are overlooking the progress made in mountain grassland cover conservation by 2020, which has significant socio-ecological implications to Europe. However, because the existing data near 2020 is scarce, the shifting character of mountain grasslands remains poorly characterized, and even less is known about the conservation outcomes because of different governance regimes and map uncertainty. Our study used Landsat satellite imagery over a transboundary mountain region in the northwestern Iberian Peninsula (Peneda-Gerês) to shed light on these aspects. Supervised classifications with a multiple classifier ensemble approach (MCE) were performed, with post classification comparison of maps established and bias-corrected to identify the trajectory in grassland cover, including protected and unprotected governance regimes. By analysing class-allocation (Shannon entropy), creating 95% confidence intervals for the area estimates, and evaluating the class-allocation thematic accuracy relationship, we characterized uncertainty in the findings. The bias-corrected estimates suggest that the positive progress claimed internationally by 2020 was not achieved. Our null hypothesis to declare a positive progress (at least equality in the proportion of grassland cover of 2019 and 2002) was rejected (X2 = 1972.1, df = 1, p < 0.001). The majority of grassland cover remained stable (67.1 ± 10.1 relative to 2002), but loss (−32.8 ± 7.1% relative to 2002 grasslands cover) overcame gain areas (+11.4 ± 6.6%), indicating net loss as the prevailing pattern over the transboundary study area (−21.4%). This feature prevailed at all extents of analysis (lowlands, −22.9%; mountains, −17.9%; mountains protected, −14.4%; mountains unprotected, −19.7%). The results also evidenced that mountain protected governance regimes experienced a lower decline in grassland extent compared to unprotected. Shannon entropy values were also significantly lower in correctly classified validation sites (z = −5.69, p = 0.0001, n = 708) suggesting a relationship between the quality of pixel assignment and thematic accuracy. We therefore encourage a post-2020 conservation and policy action to safeguard mountain grasslands by enhancing the role of protected governance regimes. To reduce uncertainty, grassland gain mapping requires additional remote sensing research to find the most adequate spatial and temporal data resolution to retrieve this process.

Performance Analysis of Abstract based Classification of Medical Journals using Ensemble Methods

Fetching the suitable web-based literature is one of the leading issues faced by the researchers now-a-days. This is due to the increase in number of articles with lack of appropriate keyword searching procedures. Specifically, in the field of biomedical research, the scientific literature repositories are growing exponentially. So, getting the appropriate and relevant documents from the vast amount of biomedical literatures is one of the key issues. To tackle the issue this research work proposed a model that uses topic modelling algorithm as the prior step before the implementation of ensemble algorithms to efficiently do the automatic labeling and classification respectively. National Center for Biotechnology Information’s (NCBI) - PubMed is the biggest source of peer-reviewed biological literatures for researchers and health practitioners in the field of biomedical. The dataset is collected from the NCBI’s Pubmed website. In this proposed work, we implemented and evaluated the performance of existing multiple classifier ensemble algorithms such as Bagging and Boosting. In this analysis, we have found that the Decision Tree algorithm performs well than other models.

A comparative analysis between late fusion of features approach and ensemble of multiple classifiers approach for image classification

SummaryIn recent times the late fusion of features approach for high‐level features extracted by multiple deep convolutional neural networks (DCNNs) has proven to be very effective in the computer vision field, especially for object classification problems. Pretrained DCNNs DenseNet‐121 and ResNet‐18 are retrained, keeping the number of output nodes equal to the number of classes present in the dataset. The last fully connected layers of these networks thereby get adapted to transform the high‐level features to a low‐dimensional feature map. Then these maps are fused to improve the performance of the model. On the other hand, an ensemble of multiple classifiers reduces the overfitting problem by combining multiple models prediction matrices. In this work, the prediction matrices of two Logsoftmax multiclass classifiers are combined. The feature maps for these two classifiers are extracted using pretrained DenseNet and ResNet. This study compares the late fusion of high‐level features approach and ensemble of multiple classifiers approach for object classification problems. Experimentation has been carried out on two benchmark datasets, such as CIFAR‐10 and CIFAR‐100, and it achieves 96.48% and 83.33% of test accuracy for ensemble of multiple classifiers and the late feature fusion approach. The proposed method has been compared with other deep architectures and datasets.

Ensemble of multiple CNN classifiers for HSI classification with Superpixel Smoothing

Hyperspectral Image analysis has gained much attention due to the presence of rich spectral information. Hyperspectral Image (HSI) classification is being utilized in a wide range of applications. Convolutional Neural Networks (CNN) are popularly used in the image classification tasks due to their capability of extracting spatial features from the raw image data. Creating an ensemble of multiple classifiers generates more robust and reliable classification results. In this paper, we propose an ensemble of four CNN classifiers with superpixel smoothing for the task of HSI classification. Stacked Auto-encoder is utilized to reduce the dimensionality of the hyperspectral data. A new method is suggested to derive the optimal number of features by exploiting the diversity among the classifiers. The uniform Local Binary Patterns (ULBP) are extracted from the HSI and is used along with reduced HSI data for classification. The two single-channel models take reduced HSI cubes as input. The two dual-Channel CNN models explore both ULBP patterns and HSI data simultaneously. We explore various techniques for combining the predictions of individual classifiers and choose the best one for ensembling purpose. The obtained prediction map is made to undergo superpixel based smoothing to remove most of the misclassified pixels. Experimental results on standard data sets confirm the superiority of the proposed ensemble model over the state of the art models. The advantages of superpixel smoothing after CNN classifications are also validated through numerical results and corresponding classification maps.

Anomaly Detection in Text Data Sets using Character-Level Representation

This paper proposes a character-level representation of unsupervised text data sets for anomaly detection problems. An empirical examination of the character-level text representation was conducted to demonstrate the ability to separate outlying and normal records using an ensemble of multiple classic numerical anomaly classifiers. Experimental results obtained on two different data sets confirmed the applicability of the developed unsupervised model to detect outlying instances in various real-world scenarios, providing the opportunity to quickly assess a large amount of textual data in terms of information consistency and conformity without knowledge of the data content itself.

Ensemble of Multiple Classifiers for Multilabel Classification of Plant Protein Subcellular Localization.

The accurate prediction of protein localization is a critical step in any functional genome annotation process. This paper proposes an improved strategy for protein subcellular localization prediction in plants based on multiple classifiers, to improve prediction results in terms of both accuracy and reliability. The prediction of plant protein subcellular localization is challenging because the underlying problem is not only a multiclass, but also a multilabel problem. Generally, plant proteins can be found in 10–14 locations/compartments. The number of proteins in some compartments (nucleus, cytoplasm, and mitochondria) is generally much greater than that in other compartments (vacuole, peroxisome, Golgi, and cell wall). Therefore, the problem of imbalanced data usually arises. Therefore, we propose an ensemble machine learning method based on average voting among heterogeneous classifiers. We first extracted various types of features suitable for each type of protein localization to form a total of 479 feature spaces. Then, feature selection methods were used to reduce the dimensions of the features into smaller informative feature subsets. This reduced feature subset was then used to train/build three different individual models. In the process of combining the three distinct classifier models, we used an average voting approach to combine the results of these three different classifiers that we constructed to return the final probability prediction. The method could predict subcellular localizations in both single- and multilabel locations, based on the voting probability. Experimental results indicated that the proposed ensemble method could achieve correct classification with an overall accuracy of 84.58% for 11 compartments, on the basis of the testing dataset.

Ensemble of Multiple Kernel SVM Classifiers for Detection of Online Spam Reviews

The Social web contains enormous content, in which higher range of online users believe the Online Reviews for their decision making before any online purchase or using services. The online reviews written are all not true. Some of the Reviews are Spam for many reasons. Data Mining Classification provides good solution to find the Review as Spam or Ham. SVM(Support Vector Machines) classifier is the most frequently used classifier. There are proven record of achieving high performance through SVM Classifier during classification of all kinds of data especially text data for binary classification. Ensemble of Multiple Kernels such as SVM-Polynomial, SVM-Linear, SVM-Sigmoid, SVM-RBF(Radial Basis Function) of SVM improves the classifiers performance in a significant way.

Memory-Efficient, Accurate and Early Diagnosis of Diabetes Through a Machine Learning Pipeline Employing Crow Search-Based Feature Engineering and a Stacking Ensemble

The early diagnosis of diabetes helps in avoiding the major risks associated with the disorder. The proposed research involves the design of a machine learning pipeline which generates the most representative feature subset of minimal size that predicts the onset of Diabetes with highest accuracy. It employs a novel diabetes dataset which is gender-neutral and representative enough unlike the well-known PID dataset. The machine learning pipelines involve multiple feature engineering pipelines to generate a reduced feature subset which is fed into multiple heterogeneous classifiers. The feature engineering involves feature selection as well as feature extraction. The former uses the ANOVA filter and Crow Search Optimization algorithm. The latter employs the Singular Value Decomposition. The classification is performed on the preprocessed dataset using a wide range of heterogeneous classifiers like Naive Bayes’, Logistic Regression, K-Nearest Neighbor, Decision Trees, Support Vector Machine, Random Forest, AdaBoost, and GradientBoost as base learners followed by their stacking ensemble. The performance evaluation of each machine learning pipeline is done through Repeated Stratified K-fold Cross Validation using the metrics of accuracy, precision, recall, F1 Score and area under Receiver Operating Characteristic curve. For each pipeline, the number of features in the preprocessed dataset varies and the highest accuracy of 98.4% is achieved with Crow Search algorithm through a stacking ensemble of multiple heterogeneous classifiers. A comparative analysis with a recent related work on the same dataset shows that the proposed feature engineering pipelines with the same set of classifiers outperform with improved accuracy using a feature set of reduced size.

Fault diagnosis method of rolling bearing based on multiple classifier ensemble of the weighted and balanced distribution adaptation under limited sample imbalance

Aiming at the minority samples cannot be effectively diagnosed when the samples are limited and imbalanced, a multiple classifier ensemble of the weighted and balanced distribution adaptation method (MC-W-BDA) is presented to solve the rolling bearing’s fault diagnosis problem under the limited samples imbalance. We adopt random sampling to obtain enough different training sample sets whose base classifiers are trained in the Reproducing Kernel Hilbert Space. The appropriate base classifiers are integrated into strong classifiers by multiple classifier ensemble strategy to obtain the final result of classification. In addition, we propose A-distance method to automatically set the optimal parameter (balance factor) in MC-W-BDA. Experimental verification verifies the feasibility and effectiveness of proposed approach.

OIC-MCE: A Practical Land Cover Mapping Approach for Limited Samples Based on Multiple Classifier Ensemble and Iterative Classification

Land cover samples are usually the foundation for supervised classification. Unfortunately, for land cover mapping in large areas, only limited samples can be used due to the time-consuming and labor-intensive sample collection. A novel and practical Object-oriented Iterative Classification method based on Multiple Classifiers Ensemble (OIC-MCE) was proposed in this paper. It systematically integrated object-oriented segmentation, Multiple Classifier Ensemble (MCE), and Iterative Classification (IC). In this method, the initial training samples were updated self-adaptively during the iterative processes. Based on these updated training samples, the inconsistent regions (ICR) in the classification results of the MCE method were reclassified to reduce their uncertainty. Three typical case studies in the China-Pakistan Economic Corridor (CPEC) indicate that the overall accuracy of the OIC-MCE method is significantly higher than that of the single classifier. After five iterations, the overall accuracy of the OIC-MCE approach increased by 5.58%–8.38% compared to the accuracy of the traditional MCE method. The spatial distribution of newly added training samples generated by the OIC-MCE approach was relatively uniform. It was confirmed by ten repeated experiments that the OIC-MCE approach has good stability. More importantly, even if the initial sample size reduced by 65%, the quality of the final classification result based on the proposed OIC-MCE approach would not be greatly affected. Therefore, the proposed OIC-MCE approach provides a new solution for land cover mapping with limited samples. Certainly, it is also well suited for land cover mapping with abundant samples.

A Safe Semi-supervised Classification Algorithm Using Multiple Classifiers Ensemble

In order to improve the performance of semi-supervised learning, a safe semi-supervised classification algorithm using multiple classifiers ensemble (S3C-MC) is proposed. First, unlabeled samples are filtered and unlabeled samples with small ambiguity are selected for semi-supervised learning. Next, the labeled training set is sampled to multiple subsets and they generate multiple classifiers to predict the filtered unlabeled sample respectively. The predicted label is formed by multiple classifiers with weighted voting mechanism, and the weight of classifier is changing constantly according to the correctness of the prediction for unlabeled samples by classifier. Then, security verification is carried out to ensure that the classifier evolves in the direction of error reduction when the new sample is added. Only the label making classifiers error lower and having the same predictive value with the three classifiers in security verification is added into the labeled set to expand the number of labeled sets. Finally, the algorithm iterates until the unlabeled sample set is empty. The experiment is carried out on the UCI data set and the result shows that the proposed S3C-MC has good safety and has a higher classification rate.

Building Deep Random Ferns Without Backpropagation

Randomized sampling-based ensemble learning is emerging as a new alternative to deep neural networks (DNNs) because it supports diversity and locality and does not require backpropagation in the learning process. By connecting randomized weak classifiers in a layer-by-layer manner, it is possible to create models having a performance similar to that of DNNs, but with less overfitting and better generalizability than before. Therefore, in this paper, we propose a deep random ferns (d-RFs) model, in which extremely randomized ferns are connected to multilayers, allowing a high classification performance and a lightweight and fast structure. The input vector is first encoded as a transformed feature vector in the feature encoder layer and then is input to the cascade layers. The feature encoding process is similar to the DNN convolution and helps improve the performance of the final output layer. Unlike in the backpropagation paradigm, the cascade layer adjusts the number of ferns and layers required for the d-RFs adaptively, using only a small scale of data. RFs ensemble approaches have considerably fewer hyper-parameters than a DNN or deep forest model, and the complexity can be determined automatically in a data-dependent manner. In addition, experimental results show that an ensemble of multiple weak classifiers reduces the bias between models through an averaging of the weakly correlated classifiers, resulting in a better overall model. The proposed lightweight d-RFs was successfully applied to benchmark datasets and yielded a similar or better accuracy level and a smaller number of parameters and operations as compared with state-of-the-art methods.

Comparing the Area of Data Mining Algorithms in Network Intrusion Detection

The network-based intrusion detection has become common to evaluate machine learning algorithms. Although the KDD Cup’99 Dataset has class imbalance over different intrusion classes, still it plays a significant role to evaluate machine learning algorithms. In this work, we utilize the singular valued decomposition technique for feature dimension reduction. We further reconstruct the features form reduced features and the selected eigenvectors. The reconstruction loss is used to decide the intrusion class for a given network feature. The intrusion class having the smallest reconstruction loss is accepted as the intrusion class in the network for that sample. The proposed system yield 97.90% accuracy on KDD Cup’99 dataset for the stated task. We have also analyzed the system with individual intrusion categories separately. This analysis suggests having a system with the ensemble of multiple classifiers; therefore we also created a random forest classifier. The random forest classifier performs significantly better than the SVD based system. The random forest classifier achieves 99.99% accuracy for intrusion detection on the same training and testing data set.

Sea Ice Classification with First-Order Logic Refined Sliding Bagging

Ren, P.; Yu, Z.-Q.; Dong, G.S.; Wang, G.-X., and, Wei, K., 2019. Sea ice classification with first-order logic refined sliding bagging. In: Jung, H.-S.; Lee, S.; Ryu, J.-H., and Cui, T. (eds.), Advances in Remote Sensing and Geoscience Information Systems of Coastal Environments. Journal of Coastal Research, Special Issue No. 90, pp. 129-134. Coconut Creek (Florida), ISSN 0749-0208.This paper proposes an automatic framework for classifying sea ice types using remote sensing imageries. Firstly, polarization features are extracted to form polarization characteristic vectors for representing individual pixels in a remote sensing image. Secondly, a multiple classifier ensemble strategy, i.e., sliding bagging, is exploited for classifying each polarization characteristic vector into a sea ice type. The sliding bagging strategy not only avoids the limitation of one single classifier for characterizing the sea ice variability over a large-scale region, but also alleviates the data imbalance over different sea ice types. Finally, the spatial structural relationships of sea ice types are encoded with first-order logic (FOL) to refine the sea ice classification resulted from the sliding bagging scheme. Experimental evaluations on RADARSAT-2 data validate the effectiveness of the proposed framework.

Bearing Fault Diagnosis Method Based on Deep Convolutional Neural Network and Random Forest Ensemble Learning.

Recently, research on data-driven bearing fault diagnosis methods has attracted increasing attention due to the availability of massive condition monitoring data. However, most existing methods still have difficulties in learning representative features from the raw data. In addition, they assume that the feature distribution of training data in source domain is the same as that of testing data in target domain, which is invalid in many real-world bearing fault diagnosis problems. Since deep learning has the automatic feature extraction ability and ensemble learning can improve the accuracy and generalization performance of classifiers, this paper proposes a novel bearing fault diagnosis method based on deep convolutional neural network (CNN) and random forest (RF) ensemble learning. Firstly, time domain vibration signals are converted into two dimensional (2D) gray-scale images containing abundant fault information by continuous wavelet transform (CWT). Secondly, a CNN model based on LeNet-5 is built to automatically extract multi-level features that are sensitive to the detection of faults from the images. Finally, the multi-level features containing both local and global information are utilized to diagnose bearing faults by the ensemble of multiple RF classifiers. In particular, low-level features containing local characteristics and accurate details in the hidden layers are combined to improve the diagnostic performance. The effectiveness of the proposed method is validated by two sets of bearing data collected from reliance electric motor and rolling mill, respectively. The experimental results indicate that the proposed method achieves high accuracy in bearing fault diagnosis under complex operational conditions and is superior to traditional methods and standard deep learning methods.

Feature extraction in an ensemble of multiple local classifiers for fault diagnosis in industrial processes

Abstract This work proposes a classification model for fault diagnosis. In a first stage, an unsupervised clustering algorithm discovers groups of potential fault classes. Subsequently, a specialized classifier is trained for each of the clusters, thus diminishing complexity and augmenting performance. The quality of the diagnosis is further improved by combining the classifiers in an ensemble. As a benchmark, data provided by the Tennessee Eastman chemical plant simulator was used, with promising results.

Multi-label Classification: a survey

Wide use of internet generates huge data which needs proper organization leading to text categorization. Earlier it was found that a document describes one category. Soon it was realized that it can describe multiple categories simultaneously. This scenario reveals the use of multi-label classification, a supervised learning approach, which assigns a predefined set of labels to an object by looking at its characteristics. Earlier used in text categorization, but soon it became the choice of researchers for wide applications like marketing, multimedia annotation, bioinformatics. Two most common approaches for multi-label classification are transformation which takes the benefit of existing single label classifiers preceded by converting multi-label data to single label, or an adaptation which designs classifiers which handle multi-label data directly. Another popular approach is ensemble of multiple classifiers taking votes of all. Other approaches are also available namely algorithm independent and algorithm dependent approach. Based on results produced, suitable metric is used for example or label wise evaluation which depends on whether prediction is binary or ranking. Every approach offers benefits and issues like loss of label dependency in transformation, complexity in case of adaptation, improvement in results using ensemble which should be considered during design of underlying application.