A novel online sequential extreme learning machine with L2,1-norm regularization for prediction problems

Abstract

In today’s world, data is produced at a very high speed and used in a large number of prediction problems. Therefore, the sequential nature of learning algorithms is in demand for batch learning algorithms. This paper presents a novel online sequential algorithm for extreme learning machine with l2,1-norm regularization (LR21OS-ELM) to handle the real-time sequential data. Wang et al. have given ELM with l2,1-norm based regularization namely LR21-ELM. This method is a batch processing model which takes data in a single chunk. So, whenever a new chunk of data arrives the model has to be retrained which takes a lot of time and memory. The proposed sequential algorithm does not require building a new model each time data arrives. This will update the previous model with new data that will save time and memory. The l2,1-norm regularization is a structural sparse-inducing norm which is integrated with an online sequential learning algorithm to diminish the complexity of the learning model by eliminating the redundant neurons of OS-ELM model. This paper proposes an iterative bi-objective optimization algorithm to solve l2,1 norm-based minimization problem and to handle the real time sequential data. The proposed model can learn sequentially arriving data in the form of chunks where chunk size can be fixed or varying. The experimental study has been conducted on several benchmark datasets collected from different research domains to prove the generalization ability of the proposed algorithm. The obtained results show that LR21OS-ELM combines the advantages of l2,1-norm regularization and online sequential learning of data and improves the prediction performance of the system.