Passive concept drift handling via variations of learning vector quantization

Moritz Heusinger,Frank-Michael Schleif,Christoph Raab

doi:10.1007/s00521-020-05242-6

Abstract

Concept drift is a change of the underlying data distribution which occurs especially with streaming data. Besides other challenges in the field of streaming data classification, concept drift has to be addressed to obtain reliable predictions. Robust Soft Learning Vector Quantization as well as Generalized Learning Vector Quantization has already shown good performance in traditional settings and is modified in this work to handle streaming data. Further, momentum-based stochastic gradient descent techniques are applied to tackle concept drift passively due to increased learning capabilities. The proposed work is tested against common benchmark algorithms and streaming data in the field and achieved promising results.

Highlights

A key concept in machine learning is to separate the training step of the model and the evaluation phase
The Adadelta version of Robust Soft Learning Vector Quantization (RSLVQ) performs significantly better than the vanilla version
The integration of Adadelta and Adamax into RSLVQ and Generalized Learning Vector Quantization (GLVQ) leads to improvements in prediction performance over their vanilla versions

Summary

Introduction

A key concept in machine learning is to separate the training step of the model and the evaluation phase. This is not applicable in the domain of stream classification. In this field, it is assumed that data arrive continuously, making the storage of data in memory unfeasible. Not all training data are available at training time, raising the need for constantly updating the model, e.g., online or incremental. Stream classifiers are prone to concept drift of streaming data, which is a change of underlying distribution and could lead to a collapse in prediction performance. There are various types of concept drift, i.e., incremental, abrupt, gradual and reoccurring, and as a consequence, a

Methods

Results

Conclusion