Heterogeneous Ensemble Learning Research Articles

A DDoS Attack Detection Method Based on Hybrid Heterogeneous Multiclassifier Ensemble Learning

The explosive growth of network traffic and its multitype on Internet have brought new and severe challenges to DDoS attack detection. To get the higher True Negative Rate (TNR), accuracy, and precision and to guarantee the robustness, stability, and universality of detection system, in this paper, we propose a DDoS attack detection method based on hybrid heterogeneous multiclassifier ensemble learning and design a heuristic detection algorithm based on Singular Value Decomposition (SVD) to construct our detection system. Experimental results show that our detection method is excellent in TNR, accuracy, and precision. Therefore, our algorithm has good detective performance for DDoS attack. Through the comparisons with Random Forest, k-Nearest Neighbor (k-NN), and Bagging comprising the component classifiers when the three algorithms are used alone by SVD and by un-SVD, it is shown that our model is superior to the state-of-the-art attack detection techniques in system generalization ability, detection stability, and overall detection performance.

Building Up a Robust Risk Mathematical Platform to Predict Colorectal Cancer

Colorectal cancer (CRC), as a result of a multistep process and under multiple factors, is one of the most common life-threatening cancers worldwide. To identify the “high risk” populations is critical for early diagnosis and improvement of overall survival rate. Of the complicated genetic and environmental factors, which group is mostly concerning colorectal carcinogenesis remains contentious. For this reason, this study collects relatively complete information of genetic variations and environmental exposure for both CRC patients and cancer-free controls; a multimethod ensemble model for CRC-risk prediction is developed by employing such big data to train and test the model. Our results demonstrate that (1) the explored genetic and environmental biomarkers are validated to connect to the CRC by biological function- or population-based evidences, (2) the model can efficiently predict the risk of CRC after parameter optimization by the big CRC-related data, and (3) our innovated heterogeneous ensemble learning model (HELM) and generalized kernel recursive maximum correntropy (GKRMC) algorithm have high prediction power. Finally, we discuss why the HELM and GKRMC can outperform the classical regression algorithms and related subjects for future study.

Reliable prediction of anti-diabetic drug failure using a reject option

The medical care for patients with type 2 diabetes generally involves ingestion of oral hypoglycemic agents in order to lower their glucose level. When predicting the result of the medication using a classification approach, high prediction accuracy of the classifier is essential because of high misclassification costs. The application of a reject option to this approach supports more accurate prediction, allowing for human experts to examine when the classifier is unreliable to predict. In this paper, we propose a reject option framework based on heterogeneous ensemble learning through a two-phase fusion. The first phase is to calculate confidence scores, which are used to determine whether to predict, and the second phase is to derive final prediction results by fusing the outputs from multiple heterogeneous classifiers. We confirm the effectiveness of the proposed method to the anti-diabetic drug failure prediction problem through experiments on actual electronic medical records data of type 2 diabetes. The proposed method yields a better trade-off between accuracy and rejection than other reject options with statistical significance. A lower prediction error is obtained for the same degree of rejection. We obtained desirable accuracy for the anti-diabetic drug failure problem by applying the proposed reject option, which allows using the classification approach in practice. The accurate prediction of drug failure at the moment of prescription can assist clinical decisions for patients. In addition, in-depth analysis can be considered for those prescriptions that are predicted as failure or rejected.

Hyperspectral Image Analysis by Spectral–Spatial Processing and Anticipative Hybrid Extreme Rotation Forest Classification

Recent classification-oriented proposals to thematic maps building from hyperspectral images have used both semisupervised approaches and spatial information for correction of spectral classification. Semisupervised approaches enrich the training data set adding similar samples to each class, whereas spatial correction is based on the natural assumption of thematic class spatial compactness. In this paper, we propose and validate the following innovations: 1) a new spectral classifier, which is called anticipative hybrid extreme rotation forest (AHERF); 2) a spatial-spectral semisupervised approach; and 3) a final spatial classification correction step. The novel heterogeneous ensemble learning approach AHERF starts with a model selection phase, using a small subsample of the training data, in order to define a ranking-based selection probability distribution of the classifier architectures that will be used in the ensemble, so that the architecture best adapted to the data domain will be used more frequently to train individual classifiers in the ensemble. After this initial phase, AHERF trains a heterogeneous ensemble applying random rotations to bootstrapped samples of the remaining training data, aiming to obtain diversified and data-domain adapted individual classifiers. The natural assumption that spatially close pixels will most likely have highly correlated values is exploited in two phases of the process pipeline. First, semisupervised label assignment is supported by spectral similarity and spatial proximity. Unsupervised spectral similarity is detected by latent class discovery. In this paper, we use a clustering algorithm (i.e., k-means). Second, maximizing class spatial compactness removes classification errors that appear as speckle noise in the classification image. The whole approach aims to use minimal sets of labeled pixels for training, which we call the seed training data set. Testing results are computed over the entire image ground truth. For comparison, we provide results in several steps: 1) of classification by AHERF and competing classifiers built by semisupervised training and 2) after spatial correction. We validate the approach on several conventional benchmarking images, achieving results which are comparable with state-of-the-art approaches.

Forecasting Electricity Demand Using an Improved Heterogeneous Ensemble Learning Algorithm

Forecasting Electricity Demand Using an Improved Heterogeneous Ensemble Learning Algorithm

Force sensor based tool condition monitoring using a heterogeneous ensemble learning model.

Tool condition monitoring (TCM) plays an important role in improving machining efficiency and guaranteeing workpiece quality. In order to realize reliable recognition of the tool condition, a robust classifier needs to be constructed to depict the relationship between tool wear states and sensory information. However, because of the complexity of the machining process and the uncertainty of the tool wear evolution, it is hard for a single classifier to fit all the collected samples without sacrificing generalization ability. In this paper, heterogeneous ensemble learning is proposed to realize tool condition monitoring in which the support vector machine (SVM), hidden Markov model (HMM) and radius basis function (RBF) are selected as base classifiers and a stacking ensemble strategy is further used to reflect the relationship between the outputs of these base classifiers and tool wear states. Based on the heterogeneous ensemble learning classifier, an online monitoring system is constructed in which the harmonic features are extracted from force signals and a minimal redundancy and maximal relevance (mRMR) algorithm is utilized to select the most prominent features. To verify the effectiveness of the proposed method, a titanium alloy milling experiment was carried out and samples with different tool wear states were collected to build the proposed heterogeneous ensemble learning classifier. Moreover, the homogeneous ensemble learning model and majority voting strategy are also adopted to make a comparison. The analysis and comparison results show that the proposed heterogeneous ensemble learning classifier performs better in both classification accuracy and stability.

Data-based prediction of sentiments using heterogeneous model ensembles

In this paper, we present an ensemble modeling approach for sentiment analysis using machine learning algorithms. The main goal of sentiment analysis is to develop estimators that are able to identify the sentiment orientation (positive, negative, or neutral) of sentences found in any arbitrary source. The novel approach presented here relies on the analysis of the words found in sentences and the formation of large sets of heterogeneous models, i.e., binary as well as multi-class classification models that are calculated by various different machine learning methods; these models shall represent the relationship between the presence of given words (or combination of words) and sentiments. All models trained during the learning phase are applied during the test phase and the final sentiment assessment is annotated with a confidence value that specifies, how reliable the models are regarding the presented decision. In the empirical part of this paper, we show results achieved using a German corpus of Amazon recensions and a set of machine learning methods (decision trees and adaptive boosting, Gaussian processes, random forests, k-nearest neighbor classification, support vector machines and artificial neural networks with evolutionary feature and parameter optimization, and genetic programming). Using a heterogeneous model ensemble learning approach that combines multi-class classifiers as well as binary classifiers, the classification accuracy can be increased significantly and the ratio of totally wrongly classified samples (i.e., those that are assigned to the completely opposite sentiment orientation) can be decreased significantly.

A New Method to Predict Probability Distribution Based on Heterogeneous Ensemble Learning

A New Method to Predict Probability Distribution Based on Heterogeneous Ensemble Learning

SpamCooling: A Parallel Heterogeneous Ensemble Spam Filtering System Based on Active Learning Techniques

Anti-spam technology is developing rapidly in recent years. With the emerging applications of machine learning in diverse fields, researchers as well as manufacturers around the world have attempted a large number of related algorithms to prevent spam. In this paper, we designed an effective anti-spam protection system, SpamCooling, based on the mechanism of active learning and parallel heterogeneous ensemble learning techniques. The system adopts a batch method to filter spam and can be easily incorporated with existing mail clients (MUA). It can actively obtain user feedbacks for providing users with personalized spam filtering experiences. The parallel heterogeneous ensemble method can help system achieve high spam detection rate as well as low ham misclassification rate.