Incremental laplacian regularization extreme learning machine for online learning

Abstract

The past decade has witnessed an explosive growth of data streams in Internet, biometrics, remote sensing and other fields. Nowadays many supervised incremental/online learning approaches have been developed, to avoid retraining and reduce the computational complexity when data come chunk by chunk. However, these methods can’t obtain satisfied performance when the labeled samples are limited. In this paper, we propose an Incremental Laplacian Regularization Extreme Learning Machine (ILR-ELM) for semi-supervised online learning, by utilizing both labeled and unlabeled samples. Unlike most of the existing semi-supervised incremental/online learning algorithms, this paper not only proposes incremental/online learning mechanism for data chunk containing both labeled and unlabeled samples but also proposes incremental/online learning mechanism for data chunk containing only unlabeled samples. The latter case is more common in practical applications because there is usually no enough time to label the samples for continuously arriving data stream. The alternative analytical solutions of ILR-ELM for the two incremental/online learning mechanisms are also presented. The performance of ILR-ELM is evaluated on three benchmark machine learning data, and the results show that it can achieve near accurate and robust classification/regression with a small number of labeled data, and outperforms the incremental/online learning approaches. Compared with the supervised and the comparative semi-supervised incremental/online learning methods, the generalization accuracy of ILR-ELM is increased by nearly 9% and 2% respectively for the classification problem. For the regression problem, the generalization error of ILR-ELM is reduced by nearly 14% than the supervised incremental/online learning methods and 1% than compared semi-supervised incremental/online learning methods Furthermore, ILR-ELM can achieve comparable prediction to semi-supervised batch learning, using less time and Random Access Memory (RAM).