The Nyström Kernel Adagrad Least Mean Square Algorithm Using k-means Sampling

Abstract

The kernel least mean square (KLMS) using the stochastic gradient descent (SGD) method has been proposed successfully in a Gaussian noise. However, KLMS suffers from performance degradation in terms of convergence rate and filtering accuracy. To this end, an adaptive gradient (Adagrad) optimization algorithm is used to find the solution to the least mean square of error, and the Nyström method is applied into KLMS to reduce its computational and spatial complexity, generating an online Nyström kernel Adagrad least mean square (NKALMS) algorithm. To further improve the filtering accuracy of NKALMS, another online Nyström kernel Adagrad least mean square algorithm using k-means sampling (NKALMS-K) is developed in this paper. Monte Carlo simulations on time series prediction demonstrate that the NKALMS-K algorithm outperforms KLMS and its extensions in the presence of Gaussian noise from the aspects of accuracy and computational efficiency.