White Box vs. Black Box Modeling: On the Performance of Deep Learning, Random Forests, and Symbolic Regression in Solving Regression Problems

Abstract

Black box machine learning techniques are methods that produce models which are functions of the inputs and produce outputs, where the internal functioning of the model is either hidden or too complicated to be analyzed. White box modeling, on the contrary, produces models whose structure is not hidden, but can be analyzed in detail. In this paper we analyze the performance of several modern black box as well as white box machine learning methods. We use them for solving several regression and classification problems, namely a set of benchmark problems of the PBML test suite, a medical data set, and a proteomics data set. Test results show that there is no method that is clearly better than the others on the benchmark data sets, on the medical data set symbolic regression is able to find the best classifiers, and on the proteomics data set the black box modeling methods clearly find better prediction models.