Selective Ensemble Method Research Articles

A tree ensemble-based two-stage model for advanced-stage colorectal cancer survival prediction

Classification techniques have widely been applied to cancer survival prediction for predicting survival or death of patients. However, little attention has been paid to patients who are predicted to die. In this work, we consider survival prediction as a two-stage task, where the first stage is to predict whether the outcome is survival or not, and the second stage is to predict the remaining lifespan for patients whose predicted outcome is death. To this end, we propose a two-stage machine learning model to enhance cancer survival prediction. At the first stage, a tree-based imbalanced ensemble classification method is proposed for classification of the survivability of advanced-stage cancer patients. At the second stage, a selective ensemble regression method is proposed for survival time prediction, where a priori knowledge is adopted for feature selection and the mean proportion of error interval is proposed for selecting base learners. Extensive computational studies performed on colorectal cancer data from SEER database demonstrate that the proposed two-stage model can achieve a more accurate prediction compared to the one-stage regression model. The results show that the proposed classification approach can effectively handle the imbalanced survivability data, and the proposed regression method outperforms several state-of-the-art regression models.

A Novel Hybrid Ensemble Classification Approach to Candidate Well Selection for CBM Well Fracturing

For coalbed methane commercial development, hydraulic fracturing is the most common stimulation treatment. The selection of wells or layers is the first link. Fracturing with big yield potential and good effect can not only reduce fracturing risk but also offer more technological choices for reservoir stimulation. A method combined with qualitative and quantitative analysis is adopted to construct the index system of candidate wells selection according to coalbed reservoir properties and development characteristics. In order to study the complex uncertainty and unbalanced classification problem in the process of candidate well selection, a fusion of interval type 2 fuzzy logic system (IT2FLS) and extreme learning machine(ELM) based selective ensemble method is proposed in this paper. Illustrative examples are provided to show to the validity of the proposed method.

On developing an automatic threshold applied to feature selection ensembles

Feature selection ensemble methods are a recent approach aiming at adding diversity in sets of selected features, improving performance and obtaining more robust and stable results. However, using an ensemble introduces the need for an aggregation step to combine all the output methods that confirm the ensemble. Besides, when trying to improve computational efficiency, ranking methods that order all initial features are preferred, and so an additional thresholding step is also mandatory. In this work two different ensemble designs based on ranking methods are described. The main difference between them is the order in which the combination and thresholding steps are performed. In addition, a new automatic threshold based on the combination of three data complexity measures is proposed and compared with traditional thresholding approaches based on retaining a fixed percentage of features. The behavior of these methods was tested, according to the SVM classification accuracy, with satisfactory results, for three different scenarios: synthetic datasets and two types of real datasets (where sample size is much higher than feature size, and where feature size is much higher than sample size).

Business failure prediction based on two-stage selective ensemble with manifold learning algorithm and kernel-based fuzzy self-organizing map

More and more models and algorithms are used to predict business failure. Many of them are not suitable for the complicated distribution of financial data, which leads to unsatisfactory prediction performance. The manifold learning algorithm is a valid method to preprocess financial data because of the good performance on dimensionality reduction for any data distribution. The kernel-based method is introduced to improve the disadvantage of fuzzy self-organizing map (FSOM) which is the limitation of spherical data distribution. Therefore, this study adopts manifold learning algorithm to select feature subsets, and employs the kernel-based FSOM (KFSOM) to compose base classifiers, and proposes the two-stage selective ensemble model for business failure prediction (BFP). First, three manifold learning algorithms, which are Isomap, Laplacian Eigenmaps and Locally Linear Embedding, are adopted to select three feature subsets from original financial data. Then, KFSOM uses three kinds of kernel functions respectively, which are Gaussian, Polynomial and Sigmoid, to obtain three classifiers. Hence, three feature subsets are computed by KFSOMs with three kernel functions respectively to acquire nine base classifiers. Last, nine base classifiers are integrated by the two-stage selective ensemble method. In the first stage, nine base classifiers are ranked according to three standards. The stepwise forward selection approach is adopted to selectively integrate nine base classifiers according to different standards. In the second stage, three selective ensembles in the first stage are integrated again to acquire the final result. In the empirical research, this work employs financial data from Chinese listed companies to predict business failure, and makes comparative analysis with previous methods. It is the conclusion that the two-stage selective ensemble with manifold learning algorithm and KFSOM is good at predicting business failure.

A Selective Ensemble Classification Method Combining Mammography Images with Ultrasound Images for Breast Cancer Diagnosis.

Breast cancer has been one of the main diseases that threatens women's life. Early detection and diagnosis of breast cancer play an important role in reducing mortality of breast cancer. In this paper, we propose a selective ensemble method integrated with the KNN, SVM, and Naive Bayes to diagnose the breast cancer combining ultrasound images with mammography images. Our experimental results have shown that the selective classification method with an accuracy of 88.73% and sensitivity of 97.06% is efficient for breast cancer diagnosis. And indicator R presents a new way to choose the base classifier for ensemble learning.

Selective ensemble of SVDDs with Renyi entropy based diversity measure

In this paper, a novel selective ensemble strategy for support vector data description (SVDD) using the Renyi entropy based diversity measure is proposed to deal with the problem of one-class classification. In order to obtain compact classification boundary, the radius of ensemble is defined as the inner product of the vector of combination weights and the vector of the radii of SVDDs. To make the center of ensemble achieve the optimal position, the Renyi entropy of the kernelized distances between the images of samples and the center of ensemble in the high-dimensional feature space is defined as the diversity measure. Moreover, to fulfill the selective ensemble, an ℓ1-norm based regularization term is introduced into the objective function of the proposed ensemble. The optimal combination weights can be iteratively obtained by the half-quadratic optimization technique. Experimental results on two synthetic data sets and twenty benchmark data sets demonstrate that the proposed selective ensemble method is superior to the single SVDD and the other four related ensemble approaches.

Particle Swarm Optimization Based Selective Ensemble of Online Sequential Extreme Learning Machine

A novel particle swarm optimization based selective ensemble (PSOSEN) of online sequential extreme learning machine (OS-ELM) is proposed. It is based on the original OS-ELM with an adaptive selective ensemble framework. Two novel insights are proposed in this paper. First, a novel selective ensemble algorithm referred to as particle swarm optimization selective ensemble is proposed, noting that PSOSEN is a general selective ensemble method which is applicable to any learning algorithms, including batch learning and online learning. Second, an adaptive selective ensemble framework for online learning is designed to balance the accuracy and speed of the algorithm. Experiments for both regression and classification problems with UCI data sets are carried out. Comparisons between OS-ELM, simple ensemble OS-ELM (EOS-ELM), genetic algorithm based selective ensemble (GASEN) of OS-ELM, and the proposed particle swarm optimization based selective ensemble of OS-ELM empirically show that the proposed algorithm achieves good generalization performance and fast learning speed.

Predict the Tertiary Structure of Protein with Binary Tree and Ensemble Strategy

In this paper we intend to apply a new method to predict tertiary structure. Several feature extraction methods adopted are physicochemical composition, recurrence quantification analysis (RQA) , pseudo amino acid composition (PseAA) and Distance frequency. We construct the binary tree Classification model, and adopt flexible neural tree models as the classifiers. We will train a number of based classifiers through different features extraction methods for every node of binary tree, then employ the selective ensemble method to ensemble them. 640 dataset is selected to our experiment. The predict accuracy with our method on this data set is 63.58%, higher than some other methods on the 640 datasets. So, our method is feasible and effective in some extent.