Novel hybrid pair recommendations based on a large-scale comparative study of concept drift detection

Abstract

During the classification of streaming data, changes in the underlying distribution make formerly learned models insecure and imprecise, which is known as the concept drift phenomenon. Online learning derives information from a vast volume of stream data, which are usually affected by these changes in unforeseen ways and are currently generated primarily by the Internet of Things, social media applications, and the stock market. There is abundant literature focused on addressing concept drift using detectors, which essentially attempt to forecast the position of the change to improve the overall accuracy by altering the base learner. This paper presents novel hybrid pairs (classifier and detector) collected from a large-scale comparison of 15 drift detectors; drift detection method (DDM), early drift detection method (EDDM), EWMA for concept drift detection (ECDD), adaptive sliding window (ADWIN), geometrical moving average (GMA), drift detection methods based on Hoeffding’s bound (HDDMA and HDDMW), Fisher exact test drift detector (FTDD), fast Hoeffding drift detection method (FHDDM), Page–Hinkley test (PH), reactive drift detection method (RDDM), SEED, statistical test of equal proportions (STEPD), SeqDrift2, and Wilcoxon rank-sum test drift detector (WSTD) and six classifiers; Naïve Bayes (NB), Hoeffding tree (HT), Hoeffding option tree (HOT), Perceptron (P), decision stump (DS), and k-nearest neighbour (KNN), to determine and recommend the best pair in accordance with the properties of the dataset. The objective of this study is to assess the contribution of a detector to a classifier and obtain the most efficient matched pairs. Through these pairwise comparison experiments, the accuracy rates and evaluation times of the pairs, as well as their false positives, true negatives, false negatives, true positives, drift detection delay, and the MCC. Additionally, the Nemenyi test is employed to compare the pairs against other methods to identify the method(s) for which there is a statistical difference. The results of the experiments indicate that the most efficient pairs—which differed for each dataset type and size—primarily include the HDDMA, RDDM, WSTD, and FHDDM detectors.