Abstract
A non-iterative learning algorithm for artificial neural networks is an alternative to optimize the neural network parameters with extremely fast convergence time. Extreme learning machine (ELM) is one of the fastest learning algorithms based on a non-iterative method for a single hidden layer feedforward neural network (SLFN) model. ELM uses a randomization technique that requires a large number of hidden nodes to achieve the high accuracy. This leads to a large and complex model, which is slow at the inference time. Previously, we reported analytical incremental learning (AIL) algorithm, which is a compact model and a non-iterative deterministic learning algorithm, to be used as an alternative. However, AIL cannot grow its set of hidden nodes, due to the node saturation problem. Here, we describe a local sigmoid method (LSM) that is also a sufficiently compact model and a non-iterative deterministic learning algorithm to overcome both the ELM randomization and AIL node saturation problems. The LSM algorithm is based on “divide and conquer” method that divides the dataset into several subsets which are easier to optimize separately. Each subset can be associated with a local segment represented as a hidden node that preserves local information of the subset. This technique helps us to understand the function of each hidden node of the network built. Moreover, we can use such a technique to explain the function of hidden nodes learned by backpropagation, the iterative algorithm. Based on our experimental results, LSM is more accurate than other non-iterative learning algorithms and one of the most compact models.
Highlights
Machine learning has become an active research area in recent years, especially for neural network (NN) models: a widely used algorithm to train the neural network models has been backpropagation (BP) [1]
We analyzed approximating the sigmoid hidden node features of single hidden layer feedforward neural network (SLFN) so that we can understand how the neural network trained by the BP algorithm works using local sigmoid method (LSM) segmentation, instead of treating it as a black box
We have described an LSM algorithm that addresses both problems of Extreme learning machine (ELM) randomization and analytical incremental learning (AIL) node saturation
Summary
Machine learning has become an active research area in recent years, especially for neural network (NN) models: a widely used algorithm to train the neural network models has been backpropagation (BP) [1]. I-ELM, EI-ELM, B-ELM and EB-ELM are based on projection method that is not optimal (not a least squares solution) Another way for improving ELM is constructing the optimal structure from a set of random hidden node features, such as pruned ELM (P-ELM) [20] and optimally pruned ELM (OP-ELM) [21]. The LSM generates the hidden nodes based on the characteristic of error vectors of a particular dataset, instead of choosing from a set of random hidden nodes as the ELM and its variants do. Another approach is the attribution method, which searches for the relationships between neurons [35]–[38] These knowledge extraction techniques try to find a high level representation, obtained from the learned model, that tells us about the semantic knowledge contributions from neurons, hidden layers and channel representations to an object or image.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
