Abstract
Conventionally, random forests are built from “greedy” decision trees which each consider only one split at a time during their construction. The sub-optimality of greedy implementation has been well-known, yet mainstream adoption of more sophisticated tree building algorithms has been lacking. We examine under what circumstances an implementation of less greedy decision trees actually yields outperformance. To this end, a “stepwise lookahead” variation of the random forest algorithm is presented for its ability to better uncover binary feature interdependencies. In contrast to the greedy approach, the decision trees included in this random forest algorithm, each simultaneously consider three split nodes in tiers of depth two. It is demonstrated on synthetic data and financial price time series that the lookahead version significantly outperforms the greedy one when (a) certain non-linear relationships between feature-pairs are present and (b) if the signal-to-noise ratio is particularly low. A long-short trading strategy for copper futures is then backtested by training both greedy and stepwise lookahead random forests to predict the signs of daily price returns. The resulting superior performance of the lookahead algorithm is at least partially explained by the presence of “XOR-like” relationships between long-term and short-term technical indicators. More generally, across all examined datasets, when no such relationships between features are present, performance across random forests is similar. Given its enhanced ability to understand the feature-interdependencies present in complex systems, this lookahead variation is a useful extension to the toolkit of data scientists, in particular for financial machine learning, where conditions (a) and (b) are typically met.
Highlights
Random forests are built from “greedy” decision trees which each consider only one split at a time during their construction
We first provide an overview of how vanilla greedy random forests (GRFs) and their underlying greedy decision trees (GDTs) are built
Upon examination of the DTs used in the lookahead random forests (LRFs)-based strategy, we identify the five most frequently selected feature combinations that are used by the nodes to split the data
Summary
Random forests are built from “greedy” decision trees which each consider only one split at a time during their construction. In contrast to the greedy approach, the decision trees included in this random forest algorithm, each simultaneously consider three split nodes in tiers of depth two It is demonstrated on synthetic data and financial price time series that the lookahead version significantly outperforms the greedy one when (a) certain nonlinear relationships between feature-pairs are present and (b) if the signal-to-noise ratio is low. Its success can be attributed to its ability to handle a variety of tasks with few parameters to tune, its ability to accommodate small sample sizes and high-dimensional feature spaces, its robustness to noise, and its aptness for parallelization[3] These advantages are a result of the RF’s design, as it is an ensemble of decision trees (DTs), popular structures for categorizing and sorting data (see Fig. 1a for an example). We label random forests constructed from LDTs as LRFs
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call