Sparse Ridgelet Kernel Regressor and its online sequential extreme learning

Abstract

In this paper, a Sparse Ridgelet Kernel Regressor (SRKR) is constructed by combing the ridgelet theory, the sparse theory with kernel trick. By using the dimensionality non-separable ridgelet kernels, SRKR is capable of processing the high-dimensional data more efficiently. Considering the preferable future of sequential learning over batch learning in problems where data arrive constantly and where batch learning is expensive, we exploit the new kernel method in an online setting using the sequential extreme learning scheme. The online learning algorithm of the examples, named Online Sequential Extreme Learning Algorithm (OS-ELA) is employed to rapidly produce a sequence of estimations. OS-ELA learns the training data one-by-one or chunk by chunk (with fixed or varying size), and discards them as long as the training procedure for those data is completed to keep the memory bounded in online learning. Evolution scheme is also incorporated to obtain a ‘good’ sparse regressor. Experiments are taken on some nonlinear time-series prediction problems, in which the examples are available one by one. Some comparisons are made and the experimental results show its efficiency and superiority to its counterparts.