Ensemble Selection Method Research Articles

Enhancing Cancer Classification Through Ensemble Machine Learning and Gene Selection Approaches

The vast dimensionality of gene expression data and the limited number of relevant genes necessitate the adoption of gene selection techniques. Furthermore, the choice of an efficient classifier plays a pivotal role in achieving accurate results. In this study, we employ the Minimum Redundancy Maximum Relevance (mRMR) method for gene selection, coupled with ensemble classifiers and individual classifiers like K-Nearest Neighbors (KNN) and Decision Trees (DT). A comparative analysis between two ensemble classifiers and two individual classifiers is conducted, revealing the superior performance of the ensemble classifiers. Our investigation utilizes four distinct cancer gene expression datasets to showcase the efficacy of employing ensemble classifiers and gene selection methods for cancer classification. The ensemble classifier (Bagging Classifier), in conjunction with the MRMR method selecting only the top 30 genes, achieved an impressive overall accuracy of 94% across all four employed datasets.

Evolutionary shift detection with ensemble variable selection

Abrupt environmental changes can lead to evolutionary shifts in trait evolution. Identifying these shifts is an important step in understanding the evolutionary history of phenotypes. The detection performances of different methods are influenced by many factors, including different numbers of shifts, shift sizes, where a shift occurs on a tree, and the types of phylogenetic structure. Furthermore, the model assumptions are oversimplified, so are likely to be violated in real data, which could cause the methods to fail. We perform simulations to assess the effect of these factors on the performance of shift detection methods. To make the comparisons more complete, we also propose an ensemble variable selection method (R package ELPASO) and compare it with existing methods (R packages ℓ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell$$\\end{document}1ou and PhylogeneticEM). The performances of methods are highly dependent on the selection criterion. ℓ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell$$\\end{document}1ou+pBIC is usually the most conservative method and it performs well when signal sizes are large. ℓ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell$$\\end{document}1ou+BIC is the least conservative method and it performs well when signal sizes are small. The ensemble method provides more balanced choices between those two methods. Moreover, the performances of all methods are heavily impacted by measurement error, tree reconstruction error and shifts in variance.

Cluster ensemble selection based on maximum quality-maximum diversity

Diversity and quality are two important factors that affect clustering ensemble performance. Some base clusterings are irrelevant and redundant, which decreases the performance of the clustering ensemble. By removing irrelevant and redundant clusterings, diversity and quality are increased simultaneously, which often leads to a more accurate ensemble solution. The relationship between quality and diversity as an optimization problem is a challenge. Based on the minimum redundancy-maximum relevance (mRMR) criterion, pair-wise and non-pair-wise methods are proposed. In the pair-wise method, each clustering is weighted in contrast to other base clusterings, whereas in the non-pair-wise method, virtual labeling is obtained using a consensus function, and then based on this labeling, each clustering is weighted. To evaluate the performance of these methods, several experiments were conducted on 10 real datasets, and the obtained results were compared to those of full ensembles. The results showed that the proposed methods led to a more significant performance improvement compared with full ensembles and other clustering ensemble selection methods.

Multivariate optimization of hyperspectral imaging for adulteration detection of ground beef: Towards the development of generic algorithms to predict adulterated ground beef and for digital sorting

Due to its nutritional value, and taste, minced beef meat (MBM) is highly valuable therefore is susceptible to economically motivated adulteration. High-throughput adulteration detection methods are required to guarantee the authenticity and safety of the MBM. Hyperspectral imaging (HSI) in the 400–1000 nm spectral range, in tandem with multivariate analysis, was evaluated to demonstrate the feasibility of developing generic models to detect minced beef meat adulterated with other types of meats. Partial least squares regression (PLSR) method, the ensemble Monte Carlo variable selection method (EMCVS), a range of spectral pre-treatments, and combinations of any two of them were evaluated to predict the amount of MBM in each prepared sample scanned. The beef prediction model was developed using data from MBM adulterated with minced chicken and turkey meats and validated with data from MBM adulterated with pork meat at adulteration levels ranging from 0 to 51% at approximately 3% increments. Good prediction results were obtained using the EMCVS, on the asymmetric least squares (AsLs) + Standard Normal variate (SNV) pre-treated reflectance spectra (Rp2 0.96, RMSEP 2.9% and RPD 5.4) with 23 selected wavelengths. An independent set of MBM samples adulterated with lamb and duck meat were then prepared similarly at concentrations ranging from 3 to 21% to test the developed method. Good results were obtained in the full range (R2p 0.94, RMSEp 3.92%, RPDp 3.83) and excellent prediction results (R2p 0.95, RMSEp 2.78%, RPDp 5.29) were obtained with 9 optimum wavelengths (484, 508, 593, 596, 602, 736, 739, 924 and 960 nm). PLS discriminant analysis (PLS-DA), the EMCVS and spectral pre-treatments were used to discriminate MBM from other adulterants and achieved almost perfect classification for calibration, cross-validation, and prediction using 12 selected spectral bands. This study developed a high-throughput generic minced meat adulteration prediction and discrimination method based on selected spectral bands that can be used to further develop low-cost portable sensors for the digital sorting of adulterated MBM, and demonstrates the feasibility to develop generic models to detect minced beef meat adulterated with other types of meat.

A scalable dynamic ensemble selection using fuzzy hyperboxes

Dynamic ensemble selection (DES) systems work by estimating the level of competence of each classifier from a pool of classifiers and selecting the most competent ones for the classification of a given test instance during inference time. The majority of dynamic ensemble selection (DES) methods evaluate the competence of classifiers using the K-Nearest Neighbors to the unknown query sample. However, KNN is very sensitive to local data distribution and needs to store all data in memory. Moreover, it performs several computations for each individual query sample. Thus, relying on the KNN technique hampers the use of DES approaches for large-scale problems and situations where data distributions are non-uniform. This article introduces a novel DES framework called FH-DES, which employs fuzzy hyperboxes to generate a competence map or incompetence map for each classifier. The competence map is generated from correctly classified samples to indicate the competence level of the classifier at each data point in the feature space, whereas the incompetence map, which shows regions where the classifier has low accuracy, is generated from misclassified samples. In this way, we can assess the competence or incompetence level of the classifier just by using the map without having to process previous samples. This feature results in a more accurate dynamic selection system with lower computational complexity compared to other dynamic selection methods. Moreover, we introduce several hyperbox expansion and contraction strategies that add incremental learning capability to the framework while keeping the computational cost low. Experimental results demonstrate that FH-DES achieves high classification accuracy with lower complexity than state-of-the-art dynamic selection methods. The source code for FH-DES is available at https://github.com/redavtalab/FH-DES.

Handheld and benchtop vis/NIR spectrometer combined with PLS regression for fast prediction of cocoa shell in cocoa powder

The fermented and dried cocoa beans are peeled, either before or after the roasting process, as peeled nibs are used for chocolate production, and shell content in cocoa powders may result from economically motivated adulteration (EMA), cross-contamination or misfits in equipment in the peeling process. The performance of this process is carefully evaluated, as values above 5% (w/w) of cocoa shell can directly affect the sensory quality of cocoa products. In this study chemometric methods were applied to near-infrared (NIR) spectra from a handheld (900–1700 nm) and a benchtop (400–1700 nm) spectrometers to predict cocoa shell content in cocoa powders. A total of 132 binary mixtures of cocoa powders with cocoa shell were prepared at several proportions (0 to 10% w/w). Partial least squares regression (PLSR) was used to develop the calibration models and different spectral preprocessing were investigated to improve the predictive performance of the models. The ensemble Monte Carlo variable selection (EMCVS) method was used to select the most informative spectral variables. Based on the results obtained with both benchtop (R2P = 0.939, RMSEP = 0.687% and RPDP = 4.14) and handheld (R2P = 0.876, RMSEP = 1.04% and RPDP = 2.82) spectrometers, NIR spectroscopy combined with the EMCVS method proved to be a highly accurate and reliable tool for predicting cocoa shell in cocoa powder. Even with a lower predictive performance than the benchtop spectrometer, the handheld spectrometer has potential to specify whether the amount of cocoa shell present in cocoa powders is in accordance with the Codex Alimentarius specifications.

Compiler Optimization Parameter Selection Method Based on Ensemble Learning

Iterative compilation based on machine learning can effectively predict a program’s compiler optimization parameters. Although having some limits, such as the low efficiency of optimization parameter search and prediction accuracy, machine learning-based solutions have been a frontier research field in the field of iterative compilation and have gained increasing attention. The research challenges are focused on learning algorithm selection, optimal parameter search, and program feature representation. For the existing problems, we propose an ensemble learning-based optimization parameter selection (ELOPS) method for the compiler. First, in order to further improve the optimization parameter search efficiency and accuracy, we proposed a multi-objective particle swarm optimization (PSO) algorithm to determine the optimal compiler parameters of the program. Second, we extracted the mixed features of the program through the feature-class relevance method, rather than using static or dynamic features alone. Finally, as the existing research usually uses a separate machine learning algorithm to build prediction models, an ensemble learning model using program features and optimization parameters was constructed to effectively predict compiler optimization parameters of the new program. Using standard performance evaluation corporation 2006 (SPEC2006) and NAS parallel benchmark (NPB) benchmarks as well as some typical scientific computing programs, we compared ELOPS with the existing methods. The experimental results showed that we can respectively achieve 1.29× and 1.26× speedup when using our method on two platforms, which are better results than those of existing methods.

Dynamic Ensemble Learning With Multi-View Kernel Collaborative Subspace Clustering for Hyperspectral Image Classification

Recently, a series of collaborative representation (CR) methods have attracted much attention for hyperspectral images classification. In this article, two CR-based dynamic ensemble selection (DES) methods using multiview kernel collaborative subspace clustering (MVKCSC) and random subspace MVKCSC (RSMVKCSC) are proposed. In order to combine spectral and spatial information to construct a region of competence (RoC), the multiview learning strategy is used in the general DES method. Compared with traditional clustering methods, the MVC can more effectively utilize multifeature information. Moreover, a new method of constructing the Laplacian matrix using kernel CR coefficients is proposed for clustering based on subspace clustering and CR theory. This method is called MVKCSC, which can obtain the clustering results by using kernel CR self-representation coefficients. In addition, to increase the diversity of samples, the random subspace method (RSM) and MVKCSC are combined for RMVKCSC. Moreover, the algorithm can obtain better clustering results by constraining samples and features simultaneously. The effectiveness of the proposed methods is validated using three hyperspectral data sets with few samples. The experimental results show that both DES-MVKCSC and DES-RSMVKCSC outperform their single classifier counterparts. In particular, the proposed methods provide superior performance compared with the state-of-the-art DES methods.

An Ordered Aggregation-Based Ensemble Selection Method of Lightweight Deep Neural Networks With Random Initialization

Due to the popularity of 5G connectivity and The Internet of Things sensors, deep learning algorithms are being extended to edge devices. Compared with AI(Artificial Intelligence) cloud platforms, the deployment of deep neural networks on edge devices must focus on low power consumption, low latency, stability and reliability. In recent years, the development of lightweight deep neural network architecture has provided a basis for the deployment of deep neural networks on edge devices. However, the shortcomings of deep neural networks, such as overconfidence, vulnerability to adversarial attack, and easy over fitting when samples are insufficient, still limit their applications in many fields. One of the ways to compensate for these defects is to use deep ensemble. An ordered aggregation-based ensemble selection algorithm is proposed, which uses soft-margin as the importance assessment metric to take full advantage of the diversity and complementarity of lightweight deep neural networks obtained from different initialization training, so as to improve the overall performance of multiple edge devices. The experimental results show that this algorithm has a significant improvement in generalization performance compared with random ensemble and ordered aggregation algorithms based on accuracy or diversity, and provides a new complementary idea for the deployment of lightweight deep neural networks on edge devices.

Multi-Objective Cluster Ensemble Based on Filter Refinement Scheme

Cluster ensemble improves the robustness and stability of clustering performances by utilizing multiple solutions. Although traditional cluster ensemble methods have achieved promising performances, they are not adaptive enough to cope with data sets that have multiple levels of complexities. Besides, these methods may contain noisy and redundancy members which have negative effects. To mitigate the above issues, in this paper, we propose a multi-objective filter refinement scheme (MOFRS). First, we perform various clustering methods on different representations of data to generate diverse solutions. Second, we propose a solution filter to select a proper method and reduce the number of initial partitions for a given data set. Third, four stability indices are designed to split instances into stable and unstable groups. Fourth, objective functions based on diversity and quality are utilized to quantify the goodness of base clustering solutions. Finally, we design an improvement oriented multi-objective evolutionary algorithm to optimize these objective functions. Extensive experimental results conducted on 27 real-world data sets show that MOFRS outperforms most cluster ensemble selection methods, and achieves statistically significant improvements, compared with full ensemble methods.

A dynamic ensemble selection method for bank telemarketing sales prediction

We propose a dynamic ensemble selection method, META-DES-AAP, to predict the success of bank telemarketing sales of time deposits. Unlike existing machine learning-based marketing sales prediction methods focusing only on prediction accuracy, META-DES-AAP considers the accuracy and average profit maximization. In META-DES-AAP, to consider both accuracy and average profit in the framework of dynamic ensemble selection using meta-training, a multi-objective optimization algorithm is designed to maximize the accuracy and average profit for base classifiers selection. Base classifiers suitable for each test telemarketing campaign are integrated according to the dynamic-based base classifiers integration method. Experimental results on bank telemarketing data show that META-DES-AAP achieves the best accuracy and the largest average profit when compared across several state-of-the-art machine learning methods. In addition, the factors influencing telemarketing on the average predicted probability of telemarketing success and average profit obtained by META-DES-AAP are analyzed.

A novel combined dynamic ensemble selection model for imbalanced data to detect COVID-19 from complete blood count

BackgroundAs blood testing is radiation-free, low-cost and simple to operate, some researchers use machine learning to detect COVID-19 from blood test data. However, few studies take into consideration the imbalanced data distribution, which can impair the performance of a classifier. MethodA novel combined dynamic ensemble selection (DES) method is proposed for imbalanced data to detect COVID-19 from complete blood count. This method combines data preprocessing and improved DES. Firstly, we use the hybrid synthetic minority over-sampling technique and edited nearest neighbor (SMOTE-ENN) to balance data and remove noise. Secondly, in order to improve the performance of DES, a novel hybrid multiple clustering and bagging classifier generation (HMCBCG) method is proposed to reinforce the diversity and local regional competence of candidate classifiers. ResultsThe experimental results based on three popular DES methods show that the performance of HMCBCG is better than only use bagging. HMCBCG+KNE obtains the best performance for COVID-19 screening with 99.81% accuracy, 99.86% F1, 99.78% G-mean and 99.81% AUC. ConclusionCompared to other advanced methods, our combined DES model can improve accuracy, G-mean, F1 and AUC of COVID-19 screening.

Ensemble selection with joint spectral clustering and structural sparsity

Generally, ensemble selection techniques are split into two categories: dynamic and static. Static ensemble selection selects a fixed subset of the original ensemble which improves the space complexity but is not flexible to each test instance. Dynamic ensemble selection selects base learners on-the-fly according to each test instance but it does not significantly improve the complexity. Currently, there is no ensemble selection technique that is robust to the test instances as well as improves space complexity. To narrow this gap, we propose a novel static ensemble selection method, called Ensemble Selection with Joint Spectral Clustering and Structural Sparsity. This method integrates spectral clustering and structural sparsity into a joint framework whose ensemble selection result is robust to test instances and consumes less space. Using 25 datasets from KEEL and UCI, we demonstrate the effectiveness of our proposed algorithm and its promising performance compared to that of other state-of-the-art algorithms.

Derivation and Validation of Essential Predictors and Risk Index for Early Detection of Diabetic Retinopathy Using Electronic Health Records.

Diabetic retinopathy (DR) is a leading cause for blindness among working-aged adults. The growing prevalence of diabetes urges for cost-effective tools to improve the compliance of eye examinations for early detection of DR. The objective of this research is to identify essential predictors and develop predictive technologies for DR using electronic health records. We conducted a retrospective analysis on a derivation cohort with 3749 DR and non-DR diabetic patients. In the analysis, an ensemble predictor selection method was employed to find essential predictors among 26 variables in demographics, duration of diabetes, complications and laboratory results. A predictive model and a risk index were built based on the selected, essential predictors, and then validated using another independent validation cohort with 869 DR and 6448 non-DR diabetic patients. Out of the 26 variables, 10 were identified to be essential for predicting DR. The predictive model achieved a AUC on the derivation cohort and a AUC on the validation cohort. For the risk index, the AUCs were and on the derivation and validation cohorts, respectively. The predictive technologies can provide an early warning sign that motivates patients to comply with eye examinations for early screening and potential treatments.

A Dynamic Ensemble Learning Algorithm based on K-means for ICU mortality prediction

This research proposes a Dynamic Ensemble Learning Algorithm based on K-means (DELAK) for intensive care unit (ICU) mortality prediction. Nowadays, the widely applied traditional scoring systems, which predict the mortality risk with some scores reflecting the severity of disease and physiological states of patients in ICU, have shown insufficient predictive performance when faced with large volume of data. Although taking advantage of large volume of data, single machine learning model and ensemble learning methods show an inadequate ability to make personalized predictions for each new patient. Dynamic ensemble selection (DES) methods which are widely studied in pattern recognition field can make personalized prediction but they only select a single classifier instead of ensemble of the results from a pool of classifiers. To overcome the limitations mentioned above, this research proposes an ensemble learning algorithm based on K-means sampling and distance-based dynamic ensemble. K-means sampling helps to achieve the diversity of base classifiers and the distance-based dynamic ensemble is a flexible fusion method which creates a personalized combination of results from base classifiers for each new test sample. To evaluate the performance for mortality prediction of the proposed algorithm, comprehensive experiments are conducted on MIMIC-III dataset. The experimental results show that DELAK achieves outstanding performance in term of AUROC and AUPRC compared with other six fusion strategies, traditional scoring systems, classical ensemble models, i.e. AdaBoost, Bagging and random forest, and dynamic ensemble selection methods in most mortality prediction tasks.

An Ensemble Hybrid Feature Selection Method for Neuropsychiatric Disorder Classification.

Magnetic resonance imagings (MRIs) are providing increased access to neuropsychiatric disorders that can be made available for advanced data analysis. However, the single type of data limits the ability of psychiatrists to distinguish the subclasses of this disease. In this paper, we propose an ensemble hybrid features selection method for the neuropsychiatric disorder classification. The method consists of a 3D DenseNet and a XGBoost, which are used to select the image features from structural MRI images and the phenotypic feature from phenotypic records, respectively. The hybrid feature is composed of image features and phenotypic features. The proposed method is validated in the Consortium for Neuropsychiatric Phenomics (CNP) dataset, where samples are classified into one of the four classes (healthy controls (HC), attention deficit hyperactivity disorder (ADHD), bipolar disorder (BD), and schizophrenia (SD)). Experimental results show that the hybrid feature can improve the performance of classification methods. The best accuracy of binary and multi-class classification can reach 91.22 and 78.62 percent, respectively. We analyze the importance of phenotypic features and image features in different classification tasks. The importance of the structure MRI images is highlighted by incorporating phenotypic features with image features to generate hybrid features. We also visualize the features of three neuropsychiatric disorders and analyze their locations in the brain region.

Heterogeneous ensemble selection for evolving data streams

Ensemble learning has been widely applied to both batch data classification and streaming data classification. For the latter setting, most existing ensemble systems are homogenous, which means they are generated from only one type of learning model. In contrast, by combining several types of different learning models, a heterogeneous ensemble system can achieve greater diversity among its members, which helps to improve its performance. Although heterogeneous ensemble systems have achieved many successes in the batch classification setting, it is not trivial to extend them directly to the data stream setting. In this study, we propose a novel HEterogeneous Ensemble Selection (HEES) method, which dynamically selects an appropriate subset of base classifiers to predict data under the stream setting. We are inspired by the observation that a well-chosen subset of good base classifiers may outperform the whole ensemble system. Here, we define a good candidate as one that expresses not only high predictive performance but also high confidence in its prediction. Our selection process is thus divided into two sub-processes: accurate-candidate selection and confident-candidate selection. We define an accurate candidate in the stream context as a base classifier with high accuracy over the current concept, while a confident candidate as one with a confidence score higher than a certain threshold. In the first sub-process, we employ the prequential accuracy to estimate the performance of a base classifier at a specific time, while in the latter sub-process, we propose a new measure to quantify the predictive confidence and provide a method to learn the threshold incrementally. The final ensemble is formed by taking the intersection of the sets of confident classifiers and accurate classifiers. Experiments on a wide range of data streams show that the proposed method achieves competitive performance with lower running time in comparison to the state-of-the-art online ensemble methods.

Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams

This work aims to connect two rarely combined research directions, i.e., non-stationary data stream classification and data analysis with skewed class distributions. We propose a novel framework employing stratified bagging for training base classifiers to integrate data preprocessing and dynamic ensemble selection methods for imbalanced data stream classification. The proposed approach has been evaluated based on computer experiments carried out on 135 artificially generated data streams with various imbalance ratios, label noise levels, and types of concept drift as well as on two selected real streams. Four preprocessing techniques and two dynamic selection methods, used on both bagging classifiers and base estimators levels, were considered. Experimentation results showed that, for highly imbalanced data streams, dynamic ensemble selection coupled with data preprocessing could outperform online and chunk-based state-of-art methods.

Drought projection in the Indochina Region based on the optimal ensemble subset of CMIP5 models

We explore future changes of temperature, precipitation, and drought characteristics in the Indochina Region (ICR) based on the optimal ensemble subset of global climate models (GCMs) of the Couple Model Intercomparison Project Phase 5 (CMIP5). The optimal ensemble subset is selected from 34 GCMs using an ensemble selection method by focusing on precipitation over ICR. Bias correction procedures for the optimal ensemble subset are examined for drought analysis in ICR. Based on the bias-corrected optimal ensemble subset, mean temperature in ICR is projected to increase around 1.1 °C (0.99 °C) in near future (2011–2040), 2.5 °C (1.8 °C) in mid future (2041–2070), and 4.3 °C (2.2 °C) in far future (2071–2100) time frames under representative concentration pathway 8.5 (RCP8.5) (RCP4.5) scenario. Mean precipitation decreases in the dry season and increases in the wet season. The 3-month Standardized Precipitation Evapotranspiration Index (SPEI-3) projects larger changes of drought characteristics than those of the 3-month Standardized Precipitation Index (SPI-3), especially quite large increases of drought duration, severity, and peak. Based on SPEI-3, the potential increase of severe drought hazard is expected in ICR in the far future period under both scenarios. The most drought-prone areas are detected over Thailand and Cambodia in which the drought characteristics are projected to expand to cover most parts of ICR in the mid and far future. The potentially dry condition over ICR is clearly depicted based on SPEI-3 with more reliable estimation after selecting the optimal ensemble subset and bias correction procedure.

Evolving interval-based representation for multiple classifier fusion

Designing an ensemble of classifiers is one of the popular research topics in machine learning since it can give better results than using each constituent member. Furthermore, the performance of ensemble can be improved using selection or adaptation. In the former, the optimal set of base classifiers, meta-classifier, original features, or meta-data is selected to obtain a better ensemble than using the entire classifiers and features. In the latter, the base classifiers or combining algorithms working on the outputs of the base classifiers are made to adapt to a particular problem. The adaptation here means that the parameters of these algorithms are trained to be optimal for each problem. In this study, we propose a novel evolving combining algorithm using the adaptation approach for the ensemble systems. Instead of using numerical value when computing the representation for each class, we propose to use the interval-based representation for the class. The optimal value of the representation is found through Particle Swarm Optimization. During classification, a test instance is assigned to the class with the interval-based representation that is closest to the base classifiers’ prediction. Experiments conducted on a number of popular dataset confirmed that the proposed method is better than the well-known ensemble systems using Decision Template and Sum Rule as combiner, L2-loss Linear Support Vector Machine, Multiple Layer Neural Network, and the ensemble selection methods based on GA-Meta-data, META-DES, and ACO.