Bayesian robust multi-extreme learning machine

Abstract

Outliers usually exist in the collected data due to human or instrumentation errors. Their existence makes the regression error obey a non-Gaussian distribution. So, people assume that the regression error obeys finite mixture distributions, such as mixture of Gaussians and mixture of Student’s t-distributions. This paper proposed two Bayesian robust regression models based on multiple extreme learning machines. First, the finite mixture error distributions are extended to their infinite counterparts according to the theory of stick-breaking construction, which can avoid selecting the optimal number of components in mixture models. Second, a multi-extreme learning machine, which combines many single extreme learning machines with different numbers of hidden nodes, is constructed, which can avoid the effects of different numbers of hidden nodes on the final regression performance. Besides, sparse Bayesian learning has also been derived to perform the sparse model weight and output weight inference automatically. The experimental results on artificial datasets and eight benchmark datasets show that the proposed two models outperform the other compared models under different rates of outliers. Also, they generate better multi-step ahead wind speed and traffic speed forecasts in real applications.