Attribute Pattern Weights (APW): A Scale to Detect Concept Drift in Data Stream Mining Models

Abstract

Extracting data structures from dynamic real-time data records is gaining prominence across industries. The need for massive mining of data sequences is increasingly observed in a wide range of user applications including social network platforms, banking sector, genomics, telecom sector, e-commerce and other sectors. To analyse multiple streams of data that is, for understanding rapid sequences of data flowing at continuous intervals, researchers are focusing on continuous improvements in data stream mining. Application of data mining models (like classifiers) in data streaming scenario mandates accurate detection of data distribution. Further, the model should adapt quickly to any variations in the distribution patterns to ensure the sustained performance of model predictability. Referred to as drift detection, the process can be gradual or abrupt. Extensive research has been made, designing several algorithms to accurately detect the type of drift and to adapt to shifts drift approaches. However, even the most reputed concept drift models have limited ability to adapt to both types of drift. The relationship between the adaptability and predictor variables is based on data distribution features and its sensitivity to in-built parameters. In this context, concept drift detection using attribute pattern weight (APW) is proposed here in this manuscript. Unlike the many of existing models, the proposed model is not dependent of any of the process targeted to apply on streaming data. The other significance of the proposed model is to detect the both types of concept drift that is gradual or abrupt. The experimental study that carried is evincing the scalability and robustness, and significance of the proposed model.