Online Adaptive Kernel Learning with Random Features for Large-scale Nonlinear Classification

Abstract

In the field of support vector machines, online random feature map algorithms are very important methods for large-scale nonlinear classification problems. At present, the existing methods have the following shortcomings: (1) If only the hyperplane vector is updated during learning while the random feature components are fixed, there is no guarantee that these online methods can adapt to the change of data distribution shape when the data is coming one by one. (2) When the kernel is selected improperly, the samples mapped to an inappropriate space may not be well classified. In order to overcome these shortcomings, considering the fact that iteratively updating random feature components can make data better fit in the current space and lead to the flexible adjustment of the kernel function, random features based online adaptive kernel learning (RF-OAK) is proposed for large-scale nonlinear classification problems. Theoretical analysis of the proposed algorithm is also provided. The experimental results and the Wilcoxon signed-ranks test show that in terms of test accuracy, the proposed method is significantly better than the state-of-the-art online feature mapping classification methods. Compared with the deep learning algorithms, the training time of RF-OAK is shorter. In terms of test accuracy, RF-OAK is better than online algorithm and comparable with offline algorithms.