Optimization-Based Extreme Learning Machine with Multi-kernel Learning Approach for Classification

Abstract

The optimization method based extreme learning machine (optimization-based ELM) is generalized from single-hidden-layer feed-forward neural networks (SLFNs) by making use of kernels instead of neuron-alike hidden nodes. This approach is known for its high scalability, low computational complexity, and mild optimization constrains. The multi-kernel learning (MKL) framework Simple MKL iteratively determines the combination of kernels by gradient descent wrapping a standard support vector machine (SVM) solver. Simple MKL can be applied to many kinds of supervised learning problems to receive a more stable performance with rapid convergence speed. This paper proposes a new approach: MK-ELM (multi-kernel extreme learning machine) that applies Simple MKL framework to the optimization-based ELM algorithm. The performance analysis on binary classification problems with various scales shows that MK-ELM tends to achieve the best generalization performance as well as being the most insensitive to parameters comparing to optimization-based ELM and Simple MKL. As a result, MK-ELM can be implemented in real applications easily.