Diversity measure as a new drift detection method in data streaming

Abstract

Data stream mining is an important research topic that has received increasing attention due to its use in a wide range of applications, such as sensor networks, banking, and telecommunication. A serious and challenging problem affecting data stream mining is concept drift. This problem occurs when the relation between the input data and the target variable changes over time. Several concept drift detection methods have been proposed, however; they either suffer from a high cost in terms of memory or run time or they are not fast enough in terms of detection speed. In this work, we propose a method, called diversity measure as a new drift detection method (DMDDM), which reacts rapidly to concept drift in less time and with less memory consumption. The proposed method combines one of the diversity measures, disagreement measure, known from static learning in streaming scenarios with the Page-Hinkley test and uses these calculations to detect drifts. The proposed method has been experimentally compared with ten drift detection methods in different drift scenarios using several datasets. The experiment results show that the proposed method is capable of detecting concept drifts faster than most of the compared methods with minimal consumption in terms of memory and run time.