Online Density Estimation over Streaming Data: A Local Adaptive Solution

Abstract

Accurate online density estimation is crucial to numerous applications that are prevalent with streaming data. Existing online approaches for density estimation somewhat lack prompt adaptability when facing drifting concepts, resulting in delayed or even deteriorated approximations. To alleviate this issue, in this work, we propose an adaptive local online density estimator, i.e. ALoKDE, for real-time density estimation on data streams. Two strategies, a statistical test for concept drift detection and an adaptive weighted local online density estimation when the drift occurs, are tightly integrated into ALoKDE. Specifically, using a weighted form, ALoKDE seeks to provide an unbiased estimation by factoring in the statistical hallmarks of the latest learned distribution and any potential distributional changes that could be introduced by each incoming instance. To ensure a high-precision estimate, ALoKDE integrates three key components: local sampling, optimal bandwidth selection at a temporal basis, and adaptive weighting factor determination. We further analyze the asymptotic properties of ALoKDE and derive its theoretical error bounds regarding bias, variance, MSE and MISE. Extensive comparative studies on various artificial and real-world streaming data demonstrate the efficacy of ALoKDE in online density estimation and real-time classification.