ExtraFerns: Fully Parallel Ensemble Learning Technique with Non-Greedy yet Minimal Memory Access Training

Abstract

Training machine learning models on edge devices is always a conflict with power consumption and computing cost. This paper proposes a hardware-oriented training method called extraFerns for a unique subset of decision tree ensembles, which drastically decreases memory access and optimizes each tree in parallel. The extraFerns gets the best of both worlds: extraTrees and randomFerns. As extraTrees does, it generates nodes by randomly selecting attributes and generating thresholds. After that, as randomFerns does, it builds ferns that are decision trees sharing an identical node in each depth. In contrast to other ensemble methods using greedy optimization, extraFerns try searching global optimization of each fern. The experimental results show that extraFerns requires only 4.3% and 4.1% memory access for training models with 3.0% and 1.2% accuracy drop compared with randomForest and extraTrees, respectively.