Original AdaBoost Algorithm Research Articles

DNAPred: Accurate Identification of DNA-Binding Sites from Protein Sequence by Ensembled Hyperplane-Distance-Based Support Vector Machines.

Accurate identification of protein-DNA binding sites is significant for both understanding protein function and drug design. Machine-learning-based methods have been extensively used for the prediction of protein-DNA binding sites. However, the data imbalance problem, in which the number of nonbinding residues (negative-class samples) is far larger than that of binding residues (positive-class samples), seriously restricts the performance improvements of machine-learning-based predictors. In this work, we designed a two-stage imbalanced learning algorithm, called ensembled hyperplane-distance-based support vector machines (E-HDSVM), to improve the prediction performance of protein-DNA binding sites. The first stage of E-HDSVM designs a new iterative sampling algorithm, called hyperplane-distance-based under-sampling (HD-US), to extract multiple subsets from the original imbalanced data set, each of which is used to train a support vector machine (SVM). Unlike traditional sampling algorithms, HD-US selects samples by calculating the distances between the samples and the separating hyperplane of the SVM. The second stage of E-HDSVM proposes an enhanced AdaBoost (EAdaBoost) algorithm to ensemble multiple trained SVMs. As an enhanced version of the original AdaBoost algorithm, EAdaBoost overcomes the overfitting problem. Stringent cross-validation and independent tests on benchmark data sets demonstrated the superiority of E-HDSVM over several popular imbalanced learning algorithms. Based on the proposed E-HDSVM algorithm, we further implemented a sequence-based protein-DNA binding site predictor, called DNAPred, which is freely available at http://csbio.njust.edu.cn/bioinf/dnapred/ for academic use. The computational experimental results showed that our predictor achieved an average overall accuracy of 91.7% and a Mathew's correlation coefficient of 0.395 on five benchmark data sets and outperformed several state-of-the-art sequence-based protein-DNA binding site predictors.

Internet Traffic Forecasting using Boosting LSTM Method

As Internet traffic is a kind of time series, the algorithms which are suitable for time series forecasting are also appropriate for Internet traffic. We study approaches for time series prediction firstly and then apply the algorithms to Internet traffic prediction in this paper. We present an ensemble method based on Long Short-Term Memory (LSTM) Network method for time series prediction with the aim of increasing the prediction accuracy of Internet traffic. We use AdaBoost algorithm to boost LSTM method because AdaBoost algorithm is the most widely used boosting algorithm. However, original AdaBoost algorithm is not suitable for prediction, in view of this fact, we use modified AdaBoost algorithm to be combined with LSTM in this paper. We use two data sets to experiment, and the results show that AdaBoost-LSTM algorithm is more accurate than single LSTM algorithm and even better than ARIMA in some situations. At last, we apply AdaBoost-LSTM algorithm to forecast the Internet traffic.

Cost-sensitive boosting algorithms: Do we really need them?

We provide a unifying perspective for two decades of work on cost-sensitive Boosting algorithms. When analyzing the literature 1997---2016, we find 15 distinct cost-sensitive variants of the original algorithm; each of these has its own motivation and claims to superiority--so who should we believe? In this work we critique the Boosting literature using four theoretical frameworks: Bayesian decision theory, the functional gradient descent view, margin theory, and probabilistic modelling. Our finding is that only three algorithms are fully supported--and the probabilistic model view suggests that all require their outputs to be calibrated for best performance. Experiments on 18 datasets across 21 degrees of imbalance support the hypothesis--showing that once calibrated, they perform equivalently, and outperform all others. Our final recommendation--based on simplicity, flexibility and performance--is to use the original Adaboost algorithm with a shifted decision threshold and calibrated probability estimates.

Improving over-fitting in ensemble regression by imprecise probabilities

In this paper, generalized versions of two ensemble methods for regression based on variants of the original AdaBoost algorithm are proposed. The generalization of these regression methods consists in restricting the unit simplex for the weights of the instances to a smaller set of weighting probabilities. Various imprecise statistical models can be used to obtain a restricted set of weighting probabilities, whose sizes each depend on a single parameter. For particular choices of this parameter, the proposed algorithms reduce to standard AdaBoost-based regression algorithms or to standard regression. The main advantage of the proposed algorithms compared to the basic AdaBoost-based regression methods is that they have less tendency to over-fitting, because the weights of the hard instances are restricted. Several simulations and applications furthermore indicate a better performance of the proposed regression methods in comparison with the corresponding standard regression methods.

Design for Fast Adaboost with Feature Selection

Since the original Adaboost algorithm is very time-consuming in training, we have designed an improved Adaboost, which adds a process of feature selection to the original Adaboost algorithm. After each round of training, we retain the features whose error rate to classify the samples are relatively low and remove the features with high error rate. In this way the time for the next training is reduced, and the whole algorithm is accelerated.

Using a Novel AdaBoost Algorithm and Chous Pseudo Amino Acid Composition for Predicting Protein Subcellular Localization

For a protein, an important characteristic is its location or compartment in a cell. This is because a protein has to be located in its proper position in a cell to perform its biological functions. Therefore, predicting protein subcellular location is an important and challenging task in current molecular and cellular biology. In this paper, based on AdaBoost.ME algorithm and Chou's PseAAC (pseudo amino acid composition), a new computational method was developed to identify protein subcellular location. AdaBoost.ME is an improved version of AdaBoost algorithm that can directly extend the original AdaBoost algorithm to deal with multi-class cases without the need to reduce it to multiple two-class problems. In some previous studies the conventional amino acid composition was applied to represent protein samples. In order to take into account the sequence order effects, in this study we use Chou's PseAAC to represent protein samples. To demonstrate that AdaBoost.ME is a robust and efficient model in predicting protein subcellular locations, the same protein dataset used by Cedano et al. (Journal of Molecular Biology, 1997, 266: 594-600) is adopted in this paper. It can be seen from the computed results that the accuracy achieved by our method is better than those by the methods developed by the previous investigators.

Fast Feature Value Searching for Face Detection

It would cost much and much time in face detector training using AdaBoost algorithm. An improved face detection algorithm called Rank-AdaBoost based on feature-value-division and Dual-AdaBoost based on dual-threshold are proposed to accelerate the training and improve detection performance. Using the improved AdaBoost, the feature values with respect to each Haar-like feature are rearrange to a definite number of ranks.The number of ranks is much less than that of the training samples, so that the test time on each training samples is saved corresponding to the original AdaBoost algorithm. Inheriting cascaded frame is also proposed here. Experimental results on MIT-CBCL face & nonface training data set illustrate that the improved algorithm could make training process convergence quickly and the training time is only one of 50 like before. Experimental results on MIT+CMU face set also show that the detection speed and accuracy are both better than the original method.

Adaptive learning approach to landmine detection

We consider landmine detection using forward-looking ground penetrating radar (FLGPR). The two main challenging tasks include extracting intricate structures of target signals and adapting a classifier to the surrounding environment through learning. Through the time-frequency (TF) analysis, we find that the most discriminant information is TF localized. This observation motivates us to use the over-complete wavelet packet transform (WPT) to sparsely represent signals with the discriminant information encoded into several bases. Then the sequential floating forward selection method is used to extract these components and thereby a neural network (NNW) classifier is designed. To further improve the classification performance and deal with the problem of detecting mines in an unconstraint environment, the AdaBoost algorithm is used. We integrate the feature selection process into the original AdaBoost algorithm. In each iteration, AdaBoost identifies the hard-to-learn examples and a new set of features which provide the specific discriminant information for these hard samples is extracted adaptively and a new classifier is trained. Experimental results based on measured data are presented, showing that a significant improvement on the classification performance can be achieved.

Soft Margins for AdaBoost

Recently ensemble methods like ADABOOST have been applied successfully in many problems, while seemingly defying the problems of overfitting. ADABOOST rarely overfits in the low noise regime, however, we show that it clearly does so for higher noise levels. Central to the understanding of this fact is the margin distribution. ADABOOST can be viewed as a constraint gradient descent in an error function with respect to the margin. We find that ADABOOST asymptotically achieves a hard margin distribution, i.e. the algorithm concentrates its resources on a few hard-to-learn patterns that are interestingly very similar to Support Vectors. A hard margin is clearly a sub-optimal strategy in the noisy case, and regularization, in our case a “mistrust” in the data, must be introduced in the algorithm to alleviate the distortions that single difficult patterns (e.g. outliers) can cause to the margin distribution. We propose several regularization methods and generalizations of the original ADABOOST algorithm to achieve a soft margin. In particular we suggest (1) regularized ADABOOSTREG where the gradient decent is done directly with respect to the soft margin and (2) regularized linear and quadratic programming (LP/QP-) ADABOOST, where the soft margin is attained by introducing slack variables. Extensive simulations demonstrate that the proposed regularized ADABOOST-type algorithms are useful and yield competitive results for noisy data.

Multi-class AdaBoost

Boosting has been a very successful technique for solving the two-class classification problem. In going from two-class to multi-class classification, most algorithms have been restricted to reducing the multi-class classification problem to multiple two-class problems. In this paper, we propose a new algorithm that naturally extends the original AdaBoost algorithm to the multiclass case without reducing it to multiple two-class problems. Similar to AdaBoost in the twoclass case, this new algorithm combines weak classifiers and only requires the performance of each weak classifier be better than random guessing (rather than 1/2). We further provide a statistical justification for the new algorithm using a novel multi-class exponential loss function and forward stage-wise additive modeling. As shown in the paper, the new algorithm is extremely easy to implement and is highly competitive with the best currently available multi-class classification methods.