Research on Decision Forest Learning Algorithm

Limin Wang,Xiongfei Li

doi:10.5539/cis.v1n2p17

Abstract

Decision Forests are investigated for their ability to provide insight into the confidence associated with each prediction, the ensembles increase predictive accuracy over the individual decision tree model established. This paper proposed a novel “bottom-top” (BT) searching strategy to learn tree structure by combining different branches with the same root, and new branches can be created to overcome overfitting phenomenon.

Highlights

Decision tree based methods of supervised learning represent one of the most popular approaches within the AI field for dealing with classification problems
The overfitting phenomenon is a persistent problem in using decision trees for classification
Promising results were achieved using ensembles of multiple classifiers, which is under the assumption that "two heads are better than one." The decisions of multiple hypotheses are combined in ensemble learning to produce more accurate results

Summary

Introduction

Decision tree based methods of supervised learning represent one of the most popular approaches within the AI field for dealing with classification problems. Promising results were achieved using ensembles of multiple classifiers, which is under the assumption that "two (or more) heads are better than one." The decisions of multiple hypotheses are combined in ensemble learning to produce more accurate results This type of learning algorithms are called Decision Forests include Random Forests (RFs) (Lariviere et al, 2005), Random Split Trees (RSs) (Breiman, 2001), and Bootstrap Aggregating (Bagging) (Pino-Mejias et al, 2008). These trees can be formed by various methods (or by one method, but with various parameters of work), by different sub-samples of observations over one and the same phenomenon, by use of different characteristics. Some ensemble algorithms have implemented modified decision tree inference algorithms in order to generate diverse decision forests

Details of Some Related Ensemble Schemes

Construction of Forests