Adaptive kernel smoothing regression using vector quantization

Abstract

A method for performing kernel smoothing regression in an online adaptive manner is presented. The approach proposed is to apply kernel smoothing regression on an incremental estimation of the (evolving) probability distribution of the incoming data stream rather than the sequence of observations. This is achieved by performing vector quantization on the incoming stream. In addition, the kernel bandwidth is adapted online using a criterion based on the median absolute deviation estimator which can be computed efficiently online. Thus, adaptive kernel smoothing regression is computed on an evolving density estimation. The method is fast and suitable for modeling streams of data. This approach is shown to be more accurate than standard kernel smoothing regression and faster for datasets larger than a few hundred observations. Experiments performed using zero order or Nadaraya-Watson kernel regression show competitive accuracy and speed of the method as compared with well-known methods for adaptive regression, such as multivariate adaptive offline regression splines (MARS), online regression, such as online-sequential extreme learning machine (OS-ELM), and evolving intelligent systems applied to regression problems, namely dynamic evolving neural-fuzzy inference system (DENFIS) and evolving Takagi-Sugeno (eTS).