Particle Swarm Optimization Based Selective Ensemble of Online Sequential Extreme Learning Machine

Yang Liu,Yue Shen,Diya Dong,Amaury Lendasse,Bo He,Rui Nian,Tianhong Yan

doi:10.1155/2015/504120

Abstract

A novel particle swarm optimization based selective ensemble (PSOSEN) of online sequential extreme learning machine (OS-ELM) is proposed. It is based on the original OS-ELM with an adaptive selective ensemble framework. Two novel insights are proposed in this paper. First, a novel selective ensemble algorithm referred to as particle swarm optimization selective ensemble is proposed, noting that PSOSEN is a general selective ensemble method which is applicable to any learning algorithms, including batch learning and online learning. Second, an adaptive selective ensemble framework for online learning is designed to balance the accuracy and speed of the algorithm. Experiments for both regression and classification problems with UCI data sets are carried out. Comparisons between OS-ELM, simple ensemble OS-ELM (EOS-ELM), genetic algorithm based selective ensemble (GASEN) of OS-ELM, and the proposed particle swarm optimization based selective ensemble of OS-ELM empirically show that the proposed algorithm achieves good generalization performance and fast learning speed.

Highlights

Feedforward neural network is one of the most prevailing neural networks for data processing in the past decades [1, 2]
A novel selective ensemble algorithm referred to as particle swarm optimization selective ensemble is proposed, noting that particle swarm optimization based selective ensemble (PSOSEN) is a general selective ensemble method which is applicable to any learning algorithms, including batch learning and online learning
PSOSEN showed its higher accuracy than the original OS-extreme learning machine (ELM) and simple ensemble of online sequential extreme learning machine (OS-ELM), which verified the feasibility of the selective ensemble method

Summary

Introduction

Feedforward neural network is one of the most prevailing neural networks for data processing in the past decades [1, 2]. ELM proves to be a few orders faster than traditional learning algorithms and obtains better generalization performance as well. It lets the fast and accurate data analytics become possible and has been applied to many fields [4,5,6]. Benefiting from the fast speed of PSO, PSOSEN is designed to be a new accurate and fast selective ensemble algorithm. Comparisons of three aspects including RMSE, standard deviation and running time between OSELM, and EOS-ELM, selective ensemble of OS-ELM (SEOSELM) with both GASEN and PSOSEN are presented.

Review of Related Work

Particle Swarm Optimization Selective Ensemble

Performance Evaluation of PSOSEN Based OS-ELM

Discussion

Conclusion