Multiple Classifier Fusion Research Articles

Intrusion Detection System with Ensemble Machine Learning Approaches using VotingClassifier

Internets have become a part of our everyday life due to the advancement in the electronics and signal processing technologies during past decades. The tremendous growth of internet leads towards the network threats. Many times firewalls and anti-viruses fails to manage the network because of this Intrusion Detection System (IDS) comes to assists us. In this paper we use IDS with Ensemble methodologies utilized in machine learning involve the fusion of multiple classifiers to improve predictive performance, while voting classifiers combine predictions from individual models to reach conclusive decisions. The paper employs a voting ensemble method combing decision tree, logistic regression and support vector machine classifier models. We test our proposedmodel to classify the NSL-KDD dataset. Our ensemble methodologies of proposed algorithmproduce a good result.

Prediction of microvascular invasion in hepatocellular carcinoma based on multi-phase dynamic enhanced CT radiomics feature and multi-classifier hierarchical fusion model

To predict microvascular invasion (MVI) in hepatocellular carcinoma (HCC) using a model based on multiphase dynamic-enhanced CT (DCE-CT) radiomics feature and hierarchical fusion of multiple classifiers. We retrospectively collected preoperative DCE-CT images from 111 patients with pathologically confirmed HCC in Guangzhou First People's Hospital between January, 2016 and April, 2020. The volume of interest was outlined in the early arterial phase, late arterial phase, portal venous phase and equilibrium phase, and radiomics features of these 4 phases were extracted. Seven classifiers based on different algorithms were trained using the filtered feature subsets to obtain multiple base classifiers under each phase. According to the hierarchical fusion strategy, a multi-criteria decision-making-based weight assignment algorithm was used for fusing each base classifier under the same phase with the model after extracting the phase information to obtain the prediction model. The proposed model was evaluated using a 5-fold cross-validation and assessed for area under the ROC curve (AUC), accuracy, sensitivity, and specificity. The prediction model was also compared with the fusion models using a single phase or multiple phases, models based on a single phase with a single classifier, models with different base classifier diversities, and 8 classifier models based on other ensemble methods. The experimental results showed that the performance of the proposed model for predicting HCCMVI was optimal after incorporating the 4 phases and 7 classifiers, with AUC, accuracy, sensitivity, and specificity of 0.828, 0.766, 0.877, and 0.648, respectively. Comparative experiments showed that this prediction model outperformed the models based on a single phase with a single classifier and other ensemble models. The proposed prediction model is effective for predicting MVI in HCC with superior performance to other models.

Dialect classification using acoustic and linguistic features in Arabic speech

<span lang="EN-US">Speech dialects refer to linguistic and pronunciation variations in the speech of the same language. Automatic dialect classification requires considerable acoustic and linguistic differences between different dialect categories of speech. This paper proposes a classification model composed of a combination of classifiers for the Arabic dialects by utilizing both the acoustic and linguistic features of spontaneous speech. The acoustic classification comprises of an ensemble of classifiers focusing on different frequency ranges within the short-term spectral features, as well as a classifier utilizing the ‘i-vector’, whilst the linguistic classifiers use features extracted by transformer models pre-trained on large Arabic text datasets. It has been shown that the proposed fusion of multiple classifiers achieves a classification accuracy of 82.44% for the identification task of five Arabic dialects. This represents the highest accuracy reported on the dataset, despite the relative simplicity of the proposed model, and has shown its applicability and relevance for dialect identification tasks. </span>

Network Intrusion Anomaly Detection Model Based on Multiclassifier Fusion Technology

With the increasing development of the industrial Internet, network security has attracted more and more attention. Among the numerous network security technologies, anomaly detection technology based on network traffic has become an important research field. At present, a large number of methods for network anomaly detection have been proposed. Most of the better performance detection methods are based on supervised machine learning algorithms, which require a large number of labelled data for model training. However, in a real network, it is impossible to manually filter and label large-scale traffic data. Network administrators can only use unsupervised machine learning algorithms for actual detection, and the detection effects are much worse than supervised learning algorithms. To improve the accuracy of the unsupervised detection methods, this study proposes a network anomaly detection model based on multiple classifier fusion technology, which applies different fusion techniques (such as Majority Vote, Weighted Majority Vote, and Naive Bayes) to fuse the detection results of the five best performing unsupervised anomaly detection algorithms. Comparative experiments are carried out on three public datasets. Experimental results show that, in terms of RECALL and AUC score, the fusion model proposed in this study achieves better performance than the five separate anomaly detection baseline algorithms, and it has better robustness and stability, which can be effectively applied to a wide range of network anomaly detection scenarios.

An Intelligent Classification Approach via Multiple Classifier Fusion and Its Application to the Fault Diagnosis

An Intelligent Classification Approach via Multiple Classifier Fusion and Its Application to the Fault Diagnosis

Motor Imagery Recognition Method Based on Multisource Transfer Learning and Multiclassifier Fusion

There are two common problems in the field of motor imagery (MI) recognition, which are poor generalization and low recognition performance. A recognition method based on multisource transfer learning and multiclassifier fusion is therefore proposed to realize the MI classification. In this approach, multisource transfer learning method is used to transfer samples from multiple source domains to target domain. The source domain selection method based on distribution similarity is designed to select those source domains whose distribution is similar to the target domain, and samples with high information entropy are selected from these source domains for transferring. Then, an MI classification method is proposed through the fusion of multiple classifiers. The classifiers are trained by labeled samples in the target domain and the transferred samples in multiple source domains. The new sample in the target domain can be identified by the weight fusion of the results of these classifiers. In order to verify the effectiveness of the proposed method, four types of motor imagery in the BCI Competition IV dataset 2a were used to evaluate the recognition ability, and the results approved an excellent recognition and generalization performance as well as a better training efficiency comparing to the well-applied methods nowadays.

Improving the Accuracy of Ensemble Machine Learning Classification Models Using a Novel Bit-Fusion Algorithm for Healthcare AI Systems.

Healthcare AI systems exclusively employ classification models for disease detection. However, with the recent research advances into this arena, it has been observed that single classification models have achieved limited accuracy in some cases. Employing fusion of multiple classifiers outputs into a single classification framework has been instrumental in achieving greater accuracy and performing automated big data analysis. The article proposes a bit fusion ensemble algorithm that minimizes the classification error rate and has been tested on various datasets. Five diversified base classifiers k- nearest neighbor (KNN), Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Decision Tree (D.T.), and Naïve Bayesian Classifier (N.B.), are used in the implementation model. Bit fusion algorithm works on the individual input from the classifiers. Decision vectors of the base classifier are weighted transformed into binary bits by comparing with high-reliability threshold parameters. The output of each base classifier is considered as soft class vectors (CV). These vectors are weighted, transformed and compared with a high threshold value of initialized δ = 0.9 for reliability. Binary patterns are extracted, and the model is trained and tested again. The standard fusion approach and proposed bit fusion algorithm have been compared by average error rate. The error rate of the Bit-fusion algorithm has been observed with the values 5.97, 12.6, 4.64, 0, 0, 27.28 for Leukemia, Breast cancer, Lung Cancer, Hepatitis, Lymphoma, Embryonal Tumors, respectively. The model is trained and tested over datasets from UCI, UEA, and UCR repositories as well which also have shown reduction in the error rates.

A SAR Target Recognition Method Based on Decision Fusion of Multiple Features and Classifiers

A synthetic aperture radar (SAR) target recognition method combining multiple features and multiple classifiers is proposed. The Zernike moments, kernel principal component analysis (KPCA), and monographic signals are used to describe SAR image features. The three types of features describe SAR target geometric shape features, projection features, and image decomposition features. Their combined use can effectively enhance the description of the target. In the classification stage, the support vector machine (SVM), sparse representation-based classification (SRC), and joint sparse representation (JSR) are used as the classifiers for the three types of features, respectively, and the corresponding decision variables are obtained. For the decision variables of the three types of features, multiple sets of weight vectors are used for weighted fusion to determine the target label of the test sample. In the experiment, based on the MSTAR dataset, experiments are performed under standard operating condition (SOC) and extended operating conditions (EOCs). The experimental results verify the effectiveness, robustness, and adaptability of the proposed method.

Hybrid intelligent technology for plant health using the fusion of evolutionary optimization and deep neural networks

AbstractIn the last decade, plant leaf disease identification has been an efficient research subject. In connection with this interest, deep learning architectures show a remarkable era in various fields of image processing and computer vision, including image classification, function detection, and image pattern recognition. In this study, we examine many aspects of convolutional neural networks for image pattern recognition. We examine the early and late fusion of multiple pattern recognition classifiers using various plant leaves. Commonly, it considers disease discovery with the diagnostic technologies available. In standard cases, planters usually do not discover the disease. Therefore, plant leaf disease detection is a significant research problem, and one of their goals is to uncover an effective way to identify leaf image disease. The article has made a potential effort to find a process that should be able to expose plant leaf disease using early and late fusion of two classifiers: modified Optimized Deep Neural Network (ODNN) with different parameters of evolutionary optimization of Grasshopper algorithm (GOA), Speeded Up Robust Features (SURF) and Convolutional Neural Network (CNN) that could support the system to achieve excellent performance. Classification quality parameters are determined, and research to explain the validation of the model has been carried out.

Locomotion Mode Recognition for Walking on Three Terrains Based on sEMG of Lower Limb and Back Muscles

Gait phase detection on different terrains is an essential procedure for amputees with a lower limb assistive device to restore walking ability. In the present study, the intent recognition of gait events on three terrains based on sEMG was presented. The class separability and robustness of time, frequency, and time-frequency domain features of sEMG signals from five leg and back muscles were quantitatively evaluated by statistical analysis to select the best features set. Then, ensemble learning method that combines the outputs of multiple classifiers into a single fusion-produced output was implemented. The results obtained from data collected from four human participants revealed that the light gradient boosting machine (LightGBM) algorithm has an average accuracy of 93.1%, a macro-F1 score of 0.929, and a calculation time of prediction of 15 ms in discriminating 12 different gait phases on three terrains. This was better than traditional voting-based multiple classifier fusion methods. LightGBM is a perfect choice for gait phase detection on different terrains in daily life.

Multiple classifiers fusion for facial expression recognition

Multiple classifiers fusion for facial expression recognition

Land-cover classification of high-resolution remote sensing image based on multi-classifier fusion and the improved Dempster–Shafer evidence theory

The use of the single machine learning classifier for high-resolution remote sensing (RS) image classification makes it difficult to improve the accuracy of classification results. To fully utilize the advantages of different classifiers for different types of ground objects, based on the Dempster–Shafer (DS) evidence theory, we propose a multi-classifier fusion method for classification of high-resolution RS images. Six machine learning classifiers: support vector machines, k-nearest neighbor, random forest, artificial neural network, classification and regression tree, and the C5.0 decision tree were selected for application in the fusion of multiple classifiers. We calculated a classifier difference index based on the accuracy and difference of the classification results of the base classifiers. Base classifiers with large differences were selected to perform integration based on the DS evidence theory. We also improved the classical DS evidence theory. First, based on the classification validity of the base classifier for different ground objects, the classification probability value of the base classifier for different samples was weight optimized. Then different fusion methods were selected according to the classification conflict coefficients between base classifiers. The results reveal that the overall accuracy and kappa coefficients of the fusion classifier are significantly better than those of the base classifier. The producer’s accuracy and user’s accuracy of the fusion results based on the improved DS evidence theory were higher than those of the fusion results based on the classical DS evidence theory.

Combining Multiple Classifiers for Domain Adaptation of Remote Sensing Image Classification

This article investigates the effectiveness of multiclassifier fusion technique on domain adaptation for remote sensing image classification. Since it is impossible to find a domain adaptation method that is optimal for different datasets, and it is also difficult to select the best base classifier for domain-invariant features, multiple domain adaptation fusion (MDAF) method and the multiple base classifier fusion (MBCF) method are proposed to achieve a more stable and superior classification performance. The most crucial step of the weighted fusion approach is to assign weights for classifiers. It is known that different classifiers have varied performances on different subsets of data, and therefore a samplewise adaptive weight is more desirable than a fixed one. For each sample, a desired weight should be able to characterize the reliability of a classifier, so that the advantages of different classifiers can be exploited. We propose a neighborhood consistency based adaptive weighting method, which assigns a large weight to a classifier on a sample if the prediction of the sample is consistent to the predictions of its local neighbors. Experiments with three remote sensing images demonstrate the efficiency of the proposed weighting strategy in the proposed MDAF and MBCF methods.

Vector score alpha integration for classifier late fusion

Alpha integration is a family of integrators that encompasses many classic fusion operators (e.g., mean, product, minimum, maximum) as particular cases. This paper proposes vector score integration (VSI), a new alpha integration method for late fusion of multiple classifiers considering the joint effect of all the classes of the multi-class problem. Theoretical derivations to optimize the parameters of VSI for achieving the minimum probability of error are provided. VSI was applied to two classification tasks using electroencephalographic signals. The first task was the automatic stage classification of a neuropsychological test performed by epileptic subjects and the second one was the classification of sleep stages from apnea patients. Four single classifiers (linear and quadratic discriminant analysis, naive Bayes, and random forest) and three competitive fusion methods were estimated for comparison: mean, majority voting, and separated score integration (SSI). SSI is based on alpha integration, but unlike the proposed method, it considers the scores from each class in isolation, not accounting for possible dependencies among scores corresponding to different classes. VSI was able to optimally combine the results from all the single classifiers, in terms of accuracy and kappa coefficient, and outperformed the results of the other fusion methods in both applications.

Evolving interval-based representation for multiple classifier fusion

Designing an ensemble of classifiers is one of the popular research topics in machine learning since it can give better results than using each constituent member. Furthermore, the performance of ensemble can be improved using selection or adaptation. In the former, the optimal set of base classifiers, meta-classifier, original features, or meta-data is selected to obtain a better ensemble than using the entire classifiers and features. In the latter, the base classifiers or combining algorithms working on the outputs of the base classifiers are made to adapt to a particular problem. The adaptation here means that the parameters of these algorithms are trained to be optimal for each problem. In this study, we propose a novel evolving combining algorithm using the adaptation approach for the ensemble systems. Instead of using numerical value when computing the representation for each class, we propose to use the interval-based representation for the class. The optimal value of the representation is found through Particle Swarm Optimization. During classification, a test instance is assigned to the class with the interval-based representation that is closest to the base classifiers’ prediction. Experiments conducted on a number of popular dataset confirmed that the proposed method is better than the well-known ensemble systems using Decision Template and Sum Rule as combiner, L2-loss Linear Support Vector Machine, Multiple Layer Neural Network, and the ensemble selection methods based on GA-Meta-data, META-DES, and ACO.

Study on Characteristic Spectrum and Multiple Classifier Fusion With Different Particle Size in Marine Sediments

Marine sediments record much information of the ecological process, which highly correlated with global and local environmental change. Particle size and its distributions of sediments indicate different ecological functions, thus are the key questions in marine ecology. The analysis method is tedious and laborious, which is not conducive for in-situ monitoring. Here, a spectral analysis was explored using surface sediments sampled in the intertidal zone of Dongdayang village, Qingdao, China. These samples were dried and sieved to pass through the mesh size of 0.3 mm, 0.2 mm, 0.1 mm, and 0.075 mm, respectively. Then, four types of subsamples were collected with the particle size of 0.3-0.2 mm, 0.2-0.1 mm, 0.1-0.075 mm, and <; 0.075 mm, respectively. The visible and near infrared reflectance spectra (226-975nm) of these subsamples with different particle size were measured. Results showed that there was a negative correlation between the spectral reflectance and the particle size. And the characteristic spectra for particle size classification were 926-975nm. These particle size were classified by the support vector machine algorithm. The classification accuracy for the calibration set and validation set was 100% and 89.06%, respectively. Furthermore, the fusion classifier was compared with the single classifier. Three spectral bands were selected as the single particle size classifier, that is 226-325nm, 826-925nm and 226-975nm. These three single classifiers were fused by voting method, forming multiple classifier fusion. A fusion classifier was recommended whose validation set had the classification accuracy of 93.75%, better than any single classifier. Multiple classifier fusion is a good tool for searching the characteristic spectra of the chemical and physical parameter in sediments. This method provides a solution for the division of particle size in sediment.

Faster R-CNN With Classifier Fusion for Automatic Detection of Small Fruits

Fruit detection is a fundamental task for automatic yield estimation. The goal is to detect all the fruits in images. The-state of the art of fruit detection algorithm, Faster R-CNN, shows a lack of detection advantage on small fruits. One of the reasons is only that single-level features and a classifier are used for localization of proposal candidates. In this article, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. During training, in order to train a model with strong generalization capability, we propose to use correlation coefficients to measure the diversity of multiple classifiers. A novel loss function with classifier correlation is introduced to train the region proposal network. We evaluate the proposed model on two data sets of small fruits. Extensive experiments show that the proposed model outperforms the state-of-the-art detectors for fruit detection. Note to Practitioners —This article was motivated by the problem of the detection of small fruits for yield estimation but it also applies to the detection of other small objects. Existing approaches to fruit detection experience difficulty in detecting small fruits and the overall detection accuracy suffers as a result. This article suggests a novel approach to detect fruits of all sizes by using a multiple classifier fusion strategy in the stage of fruit localization. In the learning of the classifiers, multilevel features of fruits are extracted. In addition, we achieve classifier diversity through the analysis of the correlation of classification probabilities. The diversity of classifiers also helps improve the performance of fruit localization. Using experiments on real fruit tree images, we show that the suggested approach is feasible on orange detection of various sites in different seasons. As a future work, we will integrate the suggested algorithm into a fruit yield estimation system and test it online.

Classifiers fusion for improved vessel recognition with application in quantification of generalized arteriolar narrowing

This paper attempts to estimate diagnostically relevant measure, i.e., Arteriovenous Ratio with an improved retinal vessel classification using feature ranking strategies and multiple classifiers decision-combination scheme. The features exploited for retinal vessel characterization are based on statistical measures of histogram, different filter responses of images and local gradient information. The feature selection process is based on two feature ranking approaches (Pearson Correlation Coefficient technique and Relief-F method) to rank the features followed by use of maximum classification accuracy of three supervised classifiers (k-Nearest Neighbor, Support Vector Machine and Naïve Bayes) as a threshold for feature subset selection. Retinal vessels are labeled using the selected feature subset and proposed hybrid classification scheme, i.e., decision fusion of multiple classifiers. The comparative analysis shows an increase in vessel classification accuracy as well as Arteriovenous Ratio calculation performance. The system is tested on three databases, a local dataset of 44 images and two publically available databases, INSPIRE-AVR containing 40 images and VICAVR containing 58 images. The local database also contains images with pathologically diseased structures. The performance of the proposed system is assessed by comparing the experimental results with the gold standard estimations as well as with the results of previous methodologies. Overall, an accuracy of 90.45%, 93.90% and 87.82% is achieved in retinal blood vessel separation with 0.0565, 0.0650 and 0.0849 mean error in Arteriovenous Ratio calculation for Local, INSPIRE-AVR and VICAVR dataset, respectively.

Electric Equipment Image Recognition Based on Decision Fusion of Multiple Classifiers for Electric Safety

This paper proposes an electric equipment image recognition algorithm based on decision fusion of multiple classifiers. Considering the drawbacks of a single classifier, three classifiers, i.e., K-nearest neighbor (KNN), support vector machine (SVM), and sparse representation-based classification (SRC) are jointly used in the classification stage. The decision values from the three classifiers are linearly combined using a weighting strategy. So, the merits of different classifiers can be fused to enhance the recognition performance. Finally, based on the fused decision values, the object label of the test sample can be decided. To validate the effectiveness of the proposed method, images of three electrical equipments (insulators, power transformers, and breakers) are classified and compared with some other methods.

Gait Recognition of Acceleration Sensor for Smart Phone Based on Multiple Classifier Fusion

Gait, as a kind of biological feature, has a profound value in personnel identification. This paper analyzes gait characteristics based on acceleration sensors of smart phones and proposes a new gait recognition method. First, in view of the existing methods in the process of extraction of gait features, a large number of redundant calculations, cycle detection error, and the phase deviation issue during the week put forward the Shape Context (SC) and Linear Time Normalized (LTN) combining SCLTN calibration method of gait cycle sequence matching, to represent the whole extract typical gait cycle gait. In view of the existing extracted gait features are still some conventional features; the velocity change of relatively uniform acceleration and the change of acceleration per unit time are proposed as new features. Secondly, combining new features with traditional features to form a new feature is set for training alternative feature set, from which the training time and recognition effect of multiple classifiers are screened. Finally, a new multiclassifier fusion method, Multiple Scale Voting (MSV), is proposed to fuse the results of Multiple classifiers to obtain the final classification results. In order to verify the performance of the proposed method, gait data of 32 testers are collected. The final experimental results show that the new feature has good separability, and the recognition rate of fusion feature set after MSV algorithm is 98.42%.