Chapter 8 - Online sequential non-tuned neural network—concept

Abstract

The extreme learning machine (ELM) is a unified framework for a single-layer feed-forward neural network (SLFFNN). In the current chapter, a new algorithm premised on the online sequential learning approach, known as the online sequential extreme learning machine (OSELM), is introduced for an SLFFNN. In general, the main learning framework of the OSELM is similar to the original ELM—the input weights and bias of hidden neurons are randomly allocated, and the output weights are analytically calculated. The difference between the algorithms of the OSELM and the original ELM is related to the calculation of the model output weight. To calculate this matrix of output weights, the ELM applied batch learning algorithms, while the OSELM considers an online learning approach. In this case, the training observations are sequentially presented to the learning algorithm, one by one or chunk-by-chunk (as a block of data). In the following section, a general background of the SLFFNN, especially the ELM, is reviewed. Following this, a detailed description of the batch learning approach utilized in the original ELM is presented, as well as the difference between batch learning and online learning. A brief review of the mathematical definition of the original ELM will be provided such that the main changes required to arrive at the OSELM are clear