Steering the interpretability of decision trees using lasso regression - an evolutionary perspective

Abstract

Since machine and deep learning have made accurate solutions possible, the search for explainable predictors has begun. Decision trees are competitive in tasks that require transparency, but have been underestimated due to their insufficient prediction performance, often caused by generalization issues. It is especially noticeable in the case of model trees, designed to solve regression tasks. Evolutionary tree induction can to some extent counteract this over and under-fitting problem with its global approach.In this paper, we examine whether integrating the lasso estimator in the tree induction process, can help to control the interpretability of the decision tree and/or improve its overall performance. We propose a new evolutionary model tree inducer called Global Lasso Tree. Its novelty lies in regularization of linear models coefficients, in the leaves during the evolutionary search. To reduce the tree's tendency to misfit, a weighted fitness function is used to dynamically balance the trade-off between conflicting objectives which is the tree error and overall complexity. The proposed method was validated on 26 publicly available regression data sets. The empirical study showed that by using the lasso-based regularization technique, we were able to steer the tree's interpretability and thus generate simpler and significantly more accurate trees.