Learning Random Model Trees for Regression

Abstract

Regression is one of the most important tasks in real-world data mining applications. Among a large number of regression models, model tree is an excellent regression model. In this paper, we single out an improved model tree algorithm via introducing randomness into the process of building model trees. We call our improved algorithm random model trees, simply RMT. RMT firstly builds an ensemble of random model trees and then averages the predictions of these random trees to predict the target value of an unseen instance. In building each random model tree, the split is selected at random from the best k splits at each non-terminal node. We experimentally test its accuracy on the 36 benchmark datasets, and compared it with some interrelated regression models. The experimental results show that RMT significantly outperforms all the other algorithms used to compare. Our work provides an effective data mining algorithm for applications especially when high-accuracy regression is required.