Novelty-aware concept drift detection for neural networks

Abstract

Neural network models are widely adopted in real-world applications for processing streaming data. However, these applications often face challenges in terms of accuracy degradation, caused by changes in the data distribution of the stream data compared to the training data. Two underlying reasons contribute to these changes. The first, known as the concept drift problem, occurs when there is a change in the correlation between the input data and the prediction output, making the models trained on the training data no longer suitable for the new data. The second reason, known as the novelty problem, arises when real-world data contains unexpected data categories that were not present in the training data, resulting in incorrect predictions. The research community has divided into different groups and each developed various methods to detect either concept drift or novelty distribution changes. However, these methods only address one aspect of the problem and are unable to distinguish between them. This leads to an inappropriate allocation of model maintenance resources, including the high cost of model retraining and the acquisition of true label data. In this study, we aim to address this gap by proposing a novel concept drift detection method that is capable of distinguishing between known labeled concept drift and novelty. Our method is also more efficient than existing drift detection methods, making it suitable for applications on neural networks.