Stream-Based Extreme Learning Machine Approach for Big Data Problems

Euler Guimarães Horta,Cristiano Leite De Castro,Antônio Pádua Braga

doi:10.1155/2015/126452

Abstract

Big Data problems demand data models with abilities to handle time-varying, massive, and high dimensional data. In this context, Active Learning emerges as an attractive technique for the development of high performance models using few data. The importance of Active Learning for Big Data becomes more evident when labeling cost is high and data is presented to the learner via data streams. This paper presents a novel Active Learning method based on Extreme Learning Machines (ELMs) and Hebbian Learning. Linearization of input data by a large size ELM hidden layer turns our method little sensitive to parameter setting. Overfitting is inherently controlled via the Hebbian Learning crosstalk term. We also demonstrate that a simple convergence test can be used as an effective labeling criterion since it points out to the amount of labels necessary for learning. The proposed method has inherent properties that make it highly attractive to handle Big Data: incremental learning via data streams, elimination of redundant patterns, and learning from a reduced informative training set. Experimental results have shown that our method is competitive with some large-margin Active Learning strategies and also with a linear SVM.

Highlights

The induction of Supervised Learning models relies on a large enough set of (xi, yi) pairs obtained by sampling xi from the input space χ according to a probability function P(X) and by querying an oracle function fg(xi) for the labels yi
SVM of Tong and Koller (SVMTK) had selected a smaller number of labels during Active Learning, its computational cost is higher than Extreme Active Learning Machine (EALM)
The results of EALM are close to those of ELM2012 which indicates that the crosstalk term of EALM has a similar regularization effect to the regularization parameter of ELM2012 but with the advantage that it is not fine-tuned because the crosstalk term is automatically controlled by our Active Learning strategy

Summary

Introduction

The induction of Supervised Learning models relies on a large enough set of (xi, yi) pairs obtained by sampling xi from the input space χ according to a probability function P(X) and by querying an oracle function fg(xi) for the labels yi. In Big Data problems, the availability of a large amount of data reveals itself as another important challenge for the induction of supervised models [1]. Dealing with Big Data requires some technique to circumvent the need of considering the entire data in the learning process. In this context, sampling probability P(X) can be controlled in order to induce good learning models using fewer patterns

Objectives

Methods

Results

Conclusion