Tri-objective optimization-based cascade ensemble pruning for deep forest

Abstract

Deep forest is a new multi-layer ensemble model, where the high time costs and storage requirements inhibit its large-scale application. However, current deep forest pruning methods used to alleviate these drawbacks do not consider its cascade coupling characteristics. Therefore, we propose a tri-objective optimization-based cascade ensemble pruning (TOOCEP) algorithm for it. Concretely, we first present a tri-objective optimization-based single-layer pruning (TOOSLP) method to prune its single-layer by simultaneously optimizing three objectives, namely accuracy, independent diversity, and coupled diversity. Particularly, the coupled diversity is designed for deep forest to deal with the coupling relationships between its adjacent layers. Then, we perform TOOSLP in a cascade framework to prune the deep forest layer-by-layer. Experimental results on 15 UCI datasets show that TOOCEP outperforms several state-of-the-art methods in accuracy and pruned rate, which significantly reduces the storage space and accelerate the prediction speed of deep forest.