Random Fourier extreme learning machine with [formula omitted]-norm regularization

Abstract

This paper proposes a novel algorithm, termed random Fourier extreme learning machine with ℓ2,1-norm regularization, to improve the robustness and compactness of the widely used extreme learning machine. In specific, we firstly introduce the random Fourier mappings as activation functions to approximate the Gaussian kernel, with the aim to improve the extendibility of the powerful kernel ELM algorithms. We then adopt the ℓ2,1-norm to eliminate the potential irrelevant neurons, resulting in a more compact and discriminative hidden layer. After that, we propose an efficient algorithm with proved convergence to solve the resultant optimization problem. Extensive experiments have been conducted on 30 benchmark data sets to compare the proposed algorithm with six popular extreme learning algorithms. As observed, our algorithm outperforms the enumerated hidden layer reinforcement algorithms. In addition, it significantly improves the computational efficiency of Gaussian kernel extreme learning machine with comparable classification and regression performance in large scale learning scenarios.