Homogeneous–Heterogeneous Hybrid Ensemble for concept-drift adaptation

Abstract

Homogeneous ensembles are very effective in concept-drift adaptation. However, choosing an appropriate base learner and its hyperparameters suitable for a stream is critical for their predictive performance. Moreover, the best base learner and its hyperparameters may change over time as the stream evolves, necessitating manual reconfiguration. On the other hand, heterogeneous ensembles train multiple base learners belonging to diverse algorithmic families with different inductive biases. Though it eliminates the need to manually choose the best base learner for a stream, their size is often restricted to the number of unique base learner algorithms, limiting their scalability. We combine the strengths of homogeneous and heterogeneous ensembles into a unified scalable ensemble framework with higher predictive performance, while eliminating the need to manually specify and adapt the optimal base learner and its hyperparameters for a stream. The proposed ensemble named H3E is a single-pass hybrid algorithm which uses a genetic algorithm (GA) based optimization in combination with stacking to provide high predictive performance at a competitive computational cost. Experiments on several real and synthetic data streams affected by diverse drift types confirm the superior predictive performance and utility of our approach in comparison to popular online ensembles.