A sequential learning method with Kalman filter and extreme learning machine for regression and time series forecasting

Abstract

In many regression and time series forecasting problems, the input data is not fully available at the beginning of the training phase. Conventional machine learning methods for batch data are not able to handle this problem. The sequential version of ELM, called Online Sequential Extreme Learning Machine (OS-ELM), addresses this problem through the least squares recursive solution for updating the network output weights. However, the implementation of OS-ELM and its extensions suffer from the problem of multicollinearity and its side effect on the variance of the weight estimates. This paper introduces a new method of sequential learning for handling the effects of multicollinearity. The proposed method, called Kalman Learning Machine (KLM), uses the Kalman filter to sequentially update the output weights of a Single Layer Feedforward Network (SLFN) based on OS-ELM. An extension of the proposed method, called Extended Kalman Learning Machine (EKLM), is presented in order to address the problem of nonlinear data. The proposed method was compared with some of the most recent and effective methods for handling the effects of multicollinearity in sequential learning problems. The experiments performed showed that the proposed method performs better than most state-of-the-art methods considering both the prediction error and training time.