Incremental Weighted Ensemble for Data Streams With Concept Drift

Abstract

As a popular strategy to tackle concept drift, chunk-based ensemble method adapts a new concept by adjusting the weights of historical classifiers. However, most previous approaches normally evaluate the historical classifier based on an entire chunk newly arrived, which may cause delayed adaptation. To address the issue, two novel ensemble models, named incremental weighted ensemble (IWE) and incremental weighted ensemble for multi-classification (IWE-M), are proposed. At each time step, all base classifiers are incrementally updated on a newly arrived instance. Following that, the instance is collected into a cache array. Once a data chunk is formed, a new base classifier is created. More specially, a forgetting mechanism based on variable-size window is designed to adjust the weight of each base classifier in IWE in terms of its classification accuracy on the latest instances in an online manner. IWE-M, an extension of IWE, aims to solve multi-class problems with local concept drifts. In IWE-M, the weight of a base classifier is expanded to a weight vector. In this way, this ensemble model can retain specific historical information about non-drift regions from a local drift. Experimental results show that the proposed ensemble frameworks outperform six competitive approaches on Accuracy and G-mean.