Hybrid Regression Tree

Abstract

A regression tree is one of the most popular machine learning-based decision models. Unlike a decision tree it predicts a continuous value. Many regression models have emerged to handle the regression problems, where most of them faced difficulties while capturing the non-linear patterns. Some regression models are sensitive to outliers, like regression trees. In this paper, a hybrid regression model is proposed, which combines the features of regression tree and ridge regression to improve the performance of regression problem. In the proposed model, the leaf nodes of the regression tree are modified. Rather than storing the mean of the corresponding targeted output values, the proposed hybrid model stores the suitable tuples in its leaf nodes. When some predictor values are inserted, the control transfers to the corresponding leaf node, and ridge regression is applied to the leaf node to predict the required values. In this method, the threshold value plays a vital role in deciding the number of tuples in the leaf nodes, which further affects the time complexity and mean squared error. Extensive comparative analysis has been made by comparing the performance of the proposed model with other regression models using four real-world datasets. The experimental results show that the proposed method outperforms the regression tree and ridge regression when applied individually.