The use of CART and multivariate regression trees for supervised and unsupervised feature selection

Abstract

Feature selection is a valuable technique in data analysis for information-preserving data reduction. This paper describes Classification and Regression Trees (CART) and Multivariate Regression Trees (MRT)-based approaches for both supervised and unsupervised feature selection. The well-known CART method allows to perform supervised feature selection by modeling one response variable ( y) by some explanatory variables ( x). The recently proposed CART extension, MRT can handle more than one response variable ( y). This allows to perform a supervised feature selection in the presence of more than one response variable. For unsupervised feature selection, where no response variables are available, we propose Auto-Associative Multivariate Regression Trees (AAMRT) where the original variables ( x) are not only used as explanatory variables ( x), but also as response variables ( y= x). Since (AA)MRT is grouping the objects into groups with similar response values by using explanatory variables, this means that the variables are found which are most responsible for the cluster structure in the data. We will demonstrate how these approaches can improve (the detection of) the cluster structure in data and how they can be used for knowledge discovery.