Tree-based ensembles for multi-output regression: Comparing multivariate approaches with separate univariate ones

Abstract

Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. In particular, they are used for predicting univariate responses. In case of multiple outputs the question arises whether it is better to separately fit univariate models or directly follow a multivariate approach. For the latter, several possibilities exist that are, e.g. based on modified splitting or stopping rules for multi-output regression. These methods are compared in extensive simulations and a real data example to help in answering the primary question when to use multivariate ensemble techniques instead of univariate ones.