Multi-classifier Fusion Research Articles

Spammer detection using multi-classifier information fusion based on evidential reasoning rule

Spammer detection is essentially a process of judging the authenticity of users, and thus can be regarded as a classification problem. In order to improve the classification performance, multi-classifier information fusion is usually used to realize the automatic detection of spammers by utilizing the information from multiple classifiers. However, the existing fusion strategies do not reasonably take the uncertainty from the results of different classifiers (views) into account, and the relative importance and reliability of each classifier are not strictly distinguished. Therefore, in order to detect spammers effectively, this paper develops a novel multi-classifier information fusion model based on the evidential reasoning (ER) rule. Firstly, according to the user's characterization strategy, the base classifiers are constructed through the profile-based, content-based and behavior-based. Then, the idea of multi-classifier fusion is combined with the ER rule, and the results of base classifiers are aggregated by considering their weights and reliabilities. Extensive experimental results on the real-world dataset verify the effectiveness of the proposed model.

Motor Imagery Recognition Method Based on Multisource Transfer Learning and Multiclassifier Fusion

There are two common problems in the field of motor imagery (MI) recognition, which are poor generalization and low recognition performance. A recognition method based on multisource transfer learning and multiclassifier fusion is therefore proposed to realize the MI classification. In this approach, multisource transfer learning method is used to transfer samples from multiple source domains to target domain. The source domain selection method based on distribution similarity is designed to select those source domains whose distribution is similar to the target domain, and samples with high information entropy are selected from these source domains for transferring. Then, an MI classification method is proposed through the fusion of multiple classifiers. The classifiers are trained by labeled samples in the target domain and the transferred samples in multiple source domains. The new sample in the target domain can be identified by the weight fusion of the results of these classifiers. In order to verify the effectiveness of the proposed method, four types of motor imagery in the BCI Competition IV dataset 2a were used to evaluate the recognition ability, and the results approved an excellent recognition and generalization performance as well as a better training efficiency comparing to the well-applied methods nowadays.

Multi-Classifier Fusion Based on MI-SFFS for Cross-Subject Emotion Recognition.

With the widespread use of emotion recognition, cross-subject emotion recognition based on EEG signals has become a hot topic in affective computing. Electroencephalography (EEG) can be used to detect the brain’s electrical activity associated with different emotions. The aim of this research is to improve the accuracy by enhancing the generalization of features. A Multi-Classifier Fusion method based on mutual information with sequential forward floating selection (MI_SFFS) is proposed. The dataset used in this paper is DEAP, which is a multi-modal open dataset containing 32 EEG channels and multiple other physiological signals. First, high-dimensional features are extracted from 15 EEG channels of DEAP after using a 10 s time window for data slicing. Second, MI and SFFS are integrated as a novel feature-selection method. Then, support vector machine (SVM), k-nearest neighbor (KNN) and random forest (RF) are employed to classify positive and negative emotions to obtain the output probabilities of classifiers as weighted features for further classification. To evaluate the model performance, leave-one-out cross-validation is adopted. Finally, cross-subject classification accuracies of 0.7089, 0.7106 and 0.7361 are achieved by the SVM, KNN and RF classifiers, respectively. The results demonstrate the feasibility of the model by splicing different classifiers’ output probabilities as a portion of the weighted features.

Multi-Classifier Fusion for Open-Set Specific Emitter Identification

To safeguard the privacy and security of IoT systems, specific emitter identification is utilized to recognize device identity with hardware characteristics. In view of the growing demand for identifying unknown devices, this paper aims to discuss open-set specific emitter identification. We firstly build up a problem formulation for open-set SEI by discussing the working mechanisms of radio signals and open-set recognition. And then it is pointed out that feature coincidence is an intractable challenge in open-set SEI. The reason, accounting for this phenome, is that pretrained fingerprint feature extractors are incapable of clustering unknown device features and differentiating them from known ones. Considering that feature coincidence leads to error recognition of unknown devices, we propose to fuse multi-classifiers in the decision layer to improve accuracy and recall. Three distinct inputs and four different fusion methods are adopted in this paper to implement multi-classifier fusion. The datasets collected at Huanghua Airport demonstrate that the proposed method can avoid the coincidence of feature space and achieve higher accuracy and recall.

Blade icing detection of wind turbine based on multi-feature and multi-classifier fusion

Wind farms are usually located in high altitude areas with a high probability of ice occurrence. Blade icing has the potential to result in unexpected mechanical failures and downtimes. In order to avoid these problems, the priority we need to do is to detect blade icing accurately. For this purpose, a novel icing detection method based on multi-feature and multi-classifier fusion is proposed in this paper. Firstly, multi-feature composed of basic features and statistical features are extracted from the operational data. Significant features are then extracted by utilizing Light Gradient Boosting Machine. Secondly, a multi-classifier fusion approach is employed to build an fusion model, which aims to obtain a much more accurate estimation for the icing state. Overall, the proposed method in this paper can achieve more accurate detection on blade icing, compared with other models. This will minimize false alarms, helping wind farms manage the operations more efficiently.

Dual Aligned Siamese Dense Regression Tracker.

Anchor or anchor-free based Siamese trackers have achieved the astonishing advancement. However, their parallel regression and classification branches lack the tracked target information link and interaction, and the corresponding independent optimization maybe lead to task-misalignment, such as the reliable classification prediction with imprecisely localization and vice versa. To address this problem, we develop a general Siamese dense regression tracker (SDRT) with both task and feature alignments. It consists of two cooperative and mutual-guidance core branches: dense local regression with RepPoint representation, the global and local multi-classifier fusion with aligned features. They complement and boost each other to constrain the results with well-localized followed to also be well-classified. Specifically, a dense local regression with RepPoint representation, directly estimates and averages multiple dense local bounding box offsets for accurate localization. And then, the refined bounding boxes can be used to learn the global and local affine alignment features for reliable multi-classifier fusion. The classified scores in turn guide the assigned positive bounding boxes for the regression task. The mutual guidance operations can bridge the connection between classification and regression substantially, since the assigned labels of one task depend on the prediction quality of the other task. The proposed tracking module is general, and it can boost both the anchor or anchor-free based Siamese trackers to some extent. The extensive tracking comparisons on six tracking benchmarks verify its favorable and competitive performance over states-of-the-arts tracking modules.

A novel multi-classifier information fusion based on Dempster–Shafer theory: application to vibration-based fault detection

Achieving a high prediction rate is a crucial task in fault detection. Although various classification procedures are available, none of them can give high accuracy in all applications. Therefore, in this article, a novel multi-classifier fusion approach is developed to boost the performance of the individual classifiers. This is acquired by using Dempster–Shafer theory. However, in cases with conflicting evidences, the Dempster–Shafer theory may give counterintuitive results. In this regard, a preprocessing technique based on a new metric is devised in order to measure and mitigate the conflict between the evidences. To evaluate and validate the effectiveness of the proposed approach, the method is applied to 15 benchmarks datasets from UCI and KEEL. Furthermore, it is applied for classifying polycrystalline nickel alloy first-stage turbine blades based on their broadband vibrational response. Through statistical analysis with different noise levels, and by comparing with four state-of-the-art fusion techniques, it is shown that the proposed method improves the classification accuracy and outperforms the individual classifiers.

Land-cover classification of high-resolution remote sensing image based on multi-classifier fusion and the improved Dempster–Shafer evidence theory

The use of the single machine learning classifier for high-resolution remote sensing (RS) image classification makes it difficult to improve the accuracy of classification results. To fully utilize the advantages of different classifiers for different types of ground objects, based on the Dempster–Shafer (DS) evidence theory, we propose a multi-classifier fusion method for classification of high-resolution RS images. Six machine learning classifiers: support vector machines, k-nearest neighbor, random forest, artificial neural network, classification and regression tree, and the C5.0 decision tree were selected for application in the fusion of multiple classifiers. We calculated a classifier difference index based on the accuracy and difference of the classification results of the base classifiers. Base classifiers with large differences were selected to perform integration based on the DS evidence theory. We also improved the classical DS evidence theory. First, based on the classification validity of the base classifier for different ground objects, the classification probability value of the base classifier for different samples was weight optimized. Then different fusion methods were selected according to the classification conflict coefficients between base classifiers. The results reveal that the overall accuracy and kappa coefficients of the fusion classifier are significantly better than those of the base classifier. The producer’s accuracy and user’s accuracy of the fusion results based on the improved DS evidence theory were higher than those of the fusion results based on the classical DS evidence theory.

Combining Multiple Classifiers for Domain Adaptation of Remote Sensing Image Classification

This article investigates the effectiveness of multiclassifier fusion technique on domain adaptation for remote sensing image classification. Since it is impossible to find a domain adaptation method that is optimal for different datasets, and it is also difficult to select the best base classifier for domain-invariant features, multiple domain adaptation fusion (MDAF) method and the multiple base classifier fusion (MBCF) method are proposed to achieve a more stable and superior classification performance. The most crucial step of the weighted fusion approach is to assign weights for classifiers. It is known that different classifiers have varied performances on different subsets of data, and therefore a samplewise adaptive weight is more desirable than a fixed one. For each sample, a desired weight should be able to characterize the reliability of a classifier, so that the advantages of different classifiers can be exploited. We propose a neighborhood consistency based adaptive weighting method, which assigns a large weight to a classifier on a sample if the prediction of the sample is consistent to the predictions of its local neighbors. Experiments with three remote sensing images demonstrate the efficiency of the proposed weighting strategy in the proposed MDAF and MBCF methods.

Recognition of cobalt-rich crusts based on multi-classifier fusion in seafloor mining environments

In seafloor mining environments, ultrasonic detection is often used for underwater target recognition, and a large number of suspended particles reduce the recognition rate of single-classifier cobalt-rich crusts. In this paper, we propose a recognition method based on multi-classifier fusion (MCF). First, the kernel Fisher discriminant analysis (KFDA) method is used to reduce the multi-class feature dimension of the signal, which improves the computational efficiency. Then, the probabilistic neural network (PNN), support vector data description (SVDD), and K-nearest neighbors (KNN) classifier are designed, and the input features of the different classifiers are selected by the genetic algorithm (GA) to improve the recognition rate of classifiers. Finally, the basic probability assignment (BPA) is recalculated in combination with the accuracy of the classifier, and conflicting evidence is synthesized to realize the MCF decision. The experimental results indicate that MCF recognition surpasses single classifier recognition. The Dempster-Shafer (D-S) theory MCF identification method proposed in this paper can be effectively applied to the identification of cobalt-rich crusts in seafloor mining environments.

Knee joint vibration signal classification algorithm based on machine learning

The knee joint is the largest and most complex flexion and extension joint of the human body. It supports most of the weight of the human body during the whole body during standing or exercise. Because the knee joint has the characteristics of complex structure and large load, it is also vulnerable to damage. An effective diagnosis in the early stage of injury or lesion of the knee joint is of great help to the later treatment. At present, the commonly used knee joint examination methods have the problems of large trauma and high cost. Therefore, this paper uses machine learning technology to study the classification algorithm of knee joint vibration signal. The research results of this paper were verified by selecting the subjects to form a healthy group and a disease injury group. The experimental results show that the proposed signal denoising algorithm is superior to the traditional denoising algorithm. After analyzing several classification algorithms, the multi-classifier fusion algorithm has excellent performance in signal classification. The experimental results show that the research results can be applied to the classification of knee joint vibration signals, and then applied to the clinical diagnosis of knee joint diseases.

Research on video flame detection algorithm based on improved DS evidence theory

The detection and prevention of flame is of great value to protect people’s lives and property safety. At present, most flame detection methods use a single classifier and have achieved some results. However, a single classification algorithm has poor adaptability to fire detection in a variety of complex situations. Therefore, a multi-classifier fusion flame detection algorithm is proposed based on Dempster-Shafer (DS) evidence theory is proposed. In short, firstly four classifiers are used to classify the same flame feature, and the four classification results are fused to make preliminary decision. The four classifiers include support vector machine (SVM), K-nearest neighbor (KNN), decision tree (DT) and random forest (RF). Second, three complementary flame features are chosen, namely color, texture and shape changes. Finally, the preliminary decision results of the three features are fused to obtain the final classification result. It should be noted that when different classifiers have strong conflicts on the classification result of the same feature, the fusion rule of DS evidence theory will be invalid. To solve this problem, the DS evidence theory is improved. For the experiment, the public flame videos are collected to construct a data set including different complex scenes for algorithm verification, where the frame rate of the video is 15 or 24 frames/s and the resolution is 320 × 240. The experimental results show that there is a strong complementary among the results of different single classifier. The multi-classifier fusion algorithm can achieve better classification performance and robust performance than the single classifier by integrating the results of each classifier, and its average detection rate reaches 93.08%. In addition, for the changes of different environments, the proposed method has higher adaptability and stability than other state-of-art methods.

Spectral-spatial hyperspectral image classification based on superpixel and multi-classifier fusion

ABSTRACT Hyperspectral image classification is a challenging problem for machine learning methods due to the small number of labelled samples and high spectral variability. In this paper, to solve this problem, a novel superpixel and multi-classifier fusion (SMCF)-based classification method for hyperspectral images is proposed. This method takes full advantage of the spectral information of superpixels and the spatial information of hyperspectral images and includes the following three steps. First, superpixels are used to increase the number of training samples and their spectral diversity. Second, label propagation (LP) is used to classify the hyperspectral images. Although LP is an efficient semi-supervised classification method, the corresponding performance is poor for certain land cover types with dispersed spatial distributions. Thus, a support vector machine (SVM) classifier is introduced to classify the hyperspectral images. Finally, the results of the SVM and LP classifiers are combined using our new class-specific weighted fusion algorithm. In the experiments, we selected three widely used and real hyperspectral data sets for evaluation. The final classification performance was evaluated based on two common metrics: the overall accuracy (OA) and the Kappa coefficient. The experimental results show that the proposed SMCF method is superior to six well-known classification methods, even when only 1% or less of the labelled samples are used.

Object Tracking with Multi-Classifier Fusion Based on Compressive Sensing and Multiple Instance Learning

Object tracking is a critical research in computer vision and has attracted significant attention over the past few years. However, the traditional object tracking algorithms often suffer from the object drifting problem due to various challenging factors in complex environments such as object occlusion and background clutter. This paper proposes a robust and effective object tracking algorithm, called MCM, which combines compressive sensing and online multiple instance learning in a multi-classifier fusion framework. In this framework, we integrate the different discriminative classifiers by learning the varied and compressed feature vectors based on different random projection matrices. And then an improved online multiple instance learning mechanism SMILE is adopted, which introduces the relative similarity to select and weight the instances in the positive bag. The experiments show that the proposed algorithm can improve the performance of object tracking on the challenging video sequences.

Multiclassifier fusion based on radiomics features for the prediction of benign and malignant primary pulmonary solid nodules.

BackgroundTo test the ability of a multiclassifier model based on radiomics features to predict benign and malignant primary pulmonary solid nodules.MethodsComputed tomography (CT) images of 342 patients with primary pulmonary solid nodules confirmed by histopathology or follow-up were retrospectively analyzed. The region of interest (ROI) of the images was delineated, and the radiomics features of the lesions were extracted. The feature weight was calculated using the relief feature selection algorithm. Based on the selected features, five classifier models were constructed: support vector machine (SVM), random forest (RF), logistic regression (LR), extreme learning machine (ELM), and K-nearest neighbor (KNN). The precision, recall rate, and area under the receiver operating characteristic curve (AUC) were used to evaluate the prediction performance of each classifier. The prediction result of each classifier was first weighted, and then all the prediction results were fused to predict the nodule type of unknown images. The prediction precision, recall rate, and AUC of the fusion classifier and single classifier were compared. Cross-validation was used to evaluate the generalization of the fusion classifier, and t- and F-tests were performed on the five classifiers and fusion classifier.ResultsFor each ROI, 450 features in four major categories were extracted and were analyzed using the relief feature selection algorithm. According to the weights, 25 highly repetitive and nonredundant stable features that played a major role in pulmonary nodule classification were selected. The fusion classifier’s prediction performance (prediction precision =92.0%, AUC =0.915) was superior to those of SVM (prediction precision =75.3%, AUC =0.740), RF (prediction precision =89.1%, AUC =0.855), LR (prediction precision =68.4%, AUC =0.681), ELM (prediction precision =87.0%, AUC =0.830), and KNN (prediction precision =77.1%, AUC =0.702). The fusion classifier showed the best null hypothesis performance in the t-test (P=0.035) and F-test (P=0.036).ConclusionsThe multiclassifier fusion model based on radiomics features had high prediction value for benign and malignant primary pulmonary solid nodules.

An early risk warning system for Outward Foreign Direct Investment in Mineral Resource-based enterprises using multi-classifiers fusion

Outward foreign direct investment in mineral resource-based enterprises (OFDI-MREs) is usually a substantial long-term investment. However, as it is affected by many uncertain factors, the investment process is full of risks. In order to reduce or lessen the investment risk of enterprises and improve the scientific approach to decision-making, it is of great significance to construct an efficient early risk warning system. In this paper, a novel method which combines the coefficient of variation method, system clustering and multi-classifier fusion to early-warn the risk of OFDI-MREs is proposed. The validity of the model is verified by using 173 sample data from 42 MREs in China. The main results are as follows: First, a hierarchically-structured risk warning indicator system with 20 indicators in three dimensions is obtained with indicator reduction; Second, the risks facing OFDI-MREs is classified into four levels based on the rate of return on equity, earnings per share, and capital accumulation rate, and most of the OFDI-MREs are at high risk; Third, the proposed multi-class fusion technology based on self-organizing data mining had higher accuracy and stability than the four widely used single-classifier models (logit regression, support vector machine, neural network, Decision Tree) and the six commonly used multi-classifier fusion methods (such as majority voting, the Bayesian method, and genetic algorithm). Accordingly, some targeted policy implications are put forward in terms of institutional distance, enterprise resource and competency foundation, which may help MREs to reduce the OFDI risks and enhance their risk prevention capabilities.

Partial discharge development stage division based on multi-classifier fusion

The oil-paper insulation air gap discharge experiment was carried out by the step-up method, and the 29-dimensional discharge characteristic parameters were extracted. The oil-paper insulation air gap discharge experiment was carried out by the step-up method, and the 29-dimensional discharge characteristic parameters were extracted. According to K-means clustering, the partial discharge development stage is divided into four stages: initial discharge stage, discharge development stage, discharge stability stage and near breakdown stage. The three basic classifiers of random forest, support vector machine and back propagation neural network are trained by the credibility-based adaptive weighted voting algorithm to train a new fusion classifier. Compared with the traditional classification fusion device, the proposed classifier fusion model can effectively improve the accuracy of the identification of the discharge development stage, up to 87.5% or more.

Feasibility study of a multi-criteria decision-making based hierarchical model for multi-modality feature and multi-classifier fusion: Applications in medical prognosis prediction

Radiomics has great prospects in terms of tumour grading, diagnosis and prediction of prognosis by analysing multifaceted data from sources such as clinical treatments, medical images, and pathology. However, exploring an effective way to manage miscellaneous clinical information, as well as to select an appropriate classifier for prediction modelling, is still demanding in a practical clinical context. In this study, we propose a multi-criterion decision-making (MCDM) based classifier fusion (MCF) strategy to combine different classifiers within an MCDM framework. A hierarchical predictive scheme (H-MCF) based on the proposed MCF is also investigated to reliably link the multi-modality features and multi-classifiers. Ten public UCI datasets and two clinical datasets were used to validate the proposed MCF and H-MCF. The experimental results showed that H-MCF has superior predictive performance when compared with the traditional fusion strategies and other fusion architectures, thus demonstrating the feasibility of the proposed H-MCF in integrating information from features of diversified modalities and different classifiers.

Iterative Classifier Fusion System for the Detection of Android Malware

Malicious software (malware) pose serious challenges for security of big data. The number and complexity of malware targeting Android devices have been exponentially increasing with the ever growing popularity of Android devices. To address this problem, multi-classifier fusion systems have long been used to increase the accuracy of malware detection for personal computers. However, previously developed systems are quite large and they cannot be transferred to Android platform. To this end, we propose Iterative Classifier Fusion System (ICFS), which is a system of minimum size, since it applies a smallest possible number of classifiers. The system applies classifiers iteratively in fusion with new iterative feature selection (IFS) procedure. We carry out extensive empirical study to determine the best options to be employed in ICFS and to compare the effectiveness of ICFS with several other traditional classifiers. The experiments show that the best outcomes for Android malware detection have been obtained by the ICFS procedure using LibSVM with polynomial kernel, combined with Multilayer Perceptron and NBtree classifier and applying IFS feature selection based on Wrapper Subset Evaluator with Particle Swarm Optimization.

Prediction of rectal toxicity of radiotherapy for prostate cancer based on multi-modality feature and multi-classifiers

To evaluate rectal toxicity of radiotherapy for prostate cancer using a novel predictive model based on multi-modality and multi-classifier fusion. We retrospectively collected the clinical data from 44 prostate cancer patients receiving external beam radiation (EBRT), including the treatment data, clinical parameters, planning CT data and the treatment plans. The clinical parameter features and dosimetric features were extracted as two different modality features, and a subset of features was selected to train the 5 base classifiers (SVM, Decision Tree, K-nearest-neighbor, Random forests and XGBoost). To establish the multi-modality and multi-classifier fusion model, a multi-criteria decision-making based weight assignment algorithm was used to assign weights for each base classifier under the same modality. A repeat 5-fold cross-validation and the 4 indexes including the area under ROC curve (AUC), accuracy, sensitivity and specificity were used to evaluate the proposed model. In addition, the proposed model was compared quantitatively with different feature selection methods, different weight allocation algorithms, the model based on single mode single classifier, and two integrated models using other fusion methods. Repeated (5 times) 5-fold cross validation of the proposed model showed an accuracy of 0.78 for distinguishing toxicity from non-toxicity with an AUC of 0.83, a specificity of 0.79 and a sensitivity of 0.76. Compared with the models based on a single mode or a single classifier and other fusion models, the proposed model can more accurately predict rectal toxicity of radiotherapy for prostate cancer.