Single Layer Feedforward Neural Network Research Articles

Road Traffic Prediction Model Using Extreme Learning Machine: The Case Study of Tangier, Morocco

An efficient and credible approach to road traffic management and prediction is a crucial aspect in the Intelligent Transportation Systems (ITS). It can strongly influence the development of road structures and projects. It is also essential for route planning and traffic regulations. In this paper, we propose a hybrid model that combines extreme learning machine (ELM) and ensemble-based techniques to predict the future hourly traffic of a road section in Tangier, a city in the north of Morocco. The model was applied to a real-world historical data set extracted from fixed sensors over a 5-years period. Our approach is based on a type of Single hidden Layer Feed-forward Neural Network (SLFN) known for being a high-speed machine learning algorithm. The model was, then, compared to other well-known algorithms in the prediction literature. Experimental results demonstrated that, according to the most commonly used criteria of error measurements (RMSE, MAE, and MAPE), our model is performing better in terms of prediction accuracy. The use of Akaike’s Information Criterion technique (AIC) has also shown that the proposed model has a higher performance.

An online self-organizing modular neural network for nonlinear system modeling

Modular neural network (MNN) has distinct advantage in many fields such as pattern recognition and pattern recognition. However it is still a challenge to dynamically adjust the MNN structure for dynamic nonlinear system modeling. This paper proposes a novel online self-organizing MNN (OSOMNN) for nonlinear system modeling. In OSOMNN, an online task decomposition algorithm and a self-organizing algorithm for subnetwork are introduced. Firstly, the task decomposition algorithm is implemented by the online clustering method based on distance and local density, which can online divide the original task into several simpler subtasks. Then subnetworks with single-layer feedforward neural network are built to learn the divided subtasks. Moreover, this paper develops a self-organizing algorithm for subnetwork, which can dynamically adjust its structure and is trained by the improved online gradient method with fixed memory mechanism (FMOGM). To demonstrate the effectiveness of OSOMNN for nonlinear system modeling, experimental investigations using four benchmark nonlinear systems and the monthly sunspots time series show that OSOMNN can automatically add or merge the subnetwork modules and optimize the structure of subnetworks for nonlinear system modeling with a better generalization performance than the established alternatives.

Correntropy induced loss based sparse robust graph regularized extreme learning machine for cancer classification

BackgroundAs a machine learning method with high performance and excellent generalization ability, extreme learning machine (ELM) is gaining popularity in various studies. Various ELM-based methods for different fields have been proposed. However, the robustness to noise and outliers is always the main problem affecting the performance of ELM.ResultsIn this paper, an integrated method named correntropy induced loss based sparse robust graph regularized extreme learning machine (CSRGELM) is proposed. The introduction of correntropy induced loss improves the robustness of ELM and weakens the negative effects of noise and outliers. By using the L2,1-norm to constrain the output weight matrix, we tend to obtain a sparse output weight matrix to construct a simpler single hidden layer feedforward neural network model. By introducing the graph regularization to preserve the local structural information of the data, the classification performance of the new method is further improved. Besides, we design an iterative optimization method based on the idea of half quadratic optimization to solve the non-convex problem of CSRGELM.ConclusionsThe classification results on the benchmark dataset show that CSRGELM can obtain better classification results compared with other methods. More importantly, we also apply the new method to the classification problems of cancer samples and get a good classification effect.

K ‐Sparse extreme learning machine

Extreme learning machine (ELM) is a single layer feed-forward neural network with advantages of fast training and good generalization properties. However, when the size of the hidden layer is increased, both of these advantages are lost as the redundant information may cause overfitting. Traditional way to deal with the issue is to introduce regularisation which promote sparsity but in the output layer weight matrix. In this Letter, we proposed the use of sparsity inside the output of the hidden layer such as to use it as the only non-linearity in the hidden layer. In the proposed formulation, we use linear activation function inside the hidden layer and keep k highest activity-bearing neurons as a measure of sparsity. Using the principal component analysis, we project the resulting output layer matrix onto a low-dimensional space in order to further remove redundant and irrelevant information, and speed up the training process. In order to verify the feasibility and effectiveness of the proposed method, we test and compare it with a number of ELM variants using benchmark datasets. Compared with these methods, our results demonstrate that the proposed method achieves better accuracy performance consistently across many different benchmark datasets.

New activation functions for single layer feedforward neural network

Artificial Neural Network (ANN) is a subfield of machine learning and it has been widely used by the researchers. The attractiveness of ANNs comes from their remarkable information processing capability. An important key parameter of ANN is activation function (AF). Different AFs can significantly affect the performance of an ANN, and therefore choosing a good AF is important. This study aims to introduce some new AFs that combine the advantages of predefined AFs and perform better than them. For this purpose, we propose some new AFs, which we called generalized swish, mean-swish, ReLU-swish, triple-state swish, sigmoid-algebraic, triple-state sigmoid, exponential swish, sinc-sigmoid and derivative of sigmoid AFs. Then, we compare the proposed AFs with some well-known and recently proposed AFs. To investigate the performance of these AFs, we use four different data sets, which are simulated data, optical interconnection network data, specifications of cars, and average house costs data.

Negative results for approximation using single layer and multilayer feedforward neural networks

We prove a negative result for the approximation of functions defined on compact subsets of Rd (where d≥2) using feedforward neural networks with one hidden layer and arbitrary continuous activation function. In a nutshell, this result claims the existence of target functions that are as difficult to approximate using these neural networks as one may want. We also demonstrate an analogous result (for general d∈N) for neural networks with an arbitrary number of hidden layers, for activation functions that are either rational functions or continuous splines with finitely many pieces.

QN-Docking: An innovative molecular docking methodology based on Q-Networks

Molecular docking is often used in computational chemistry to accelerate drug discovery at early stages. Many molecular simulations are performed to select the right pharmacological candidate. However, traditional docking methods are based on optimization heuristics such as Monte Carlo or genetic that try several hundreds of these candidates giving rise to expensive computations. Thus, an alternative methodology called QN-Docking is proposed for developing docking simulations more efficiently. This new approach is built upon Q-learning using a single-layer feedforward neural network to train a single ligand or drug candidate (the agent) to find its optimal interaction with the host molecule. In addition, the corresponding Reinforcement Learning environment and the reward function based on a force-field scoring function are implemented. The proposed method is evaluated in an exemplary molecular scenario based on the kaempferol and beta-cyclodextrin. Results for the prediction phase show that QN-Docking achieves 8× speedup compared to stochastic methods such as METADOCK 2, a novel high-throughput parallel metaheuristic software for docking. Moreover, these results could be extended to many other ligand-host pairs to ultimately develop a general and faster docking method.

A Hybrid Method Based on Extreme Learning Machine and Self Organizing Map for Pattern Classification.

Extreme learning machine is a fast learning algorithm for single hidden layer feedforward neural network. However, an improper number of hidden neurons and random parameters have a great effect on the performance of the extreme learning machine. In order to select a suitable number of hidden neurons, this paper proposes a novel hybrid learning based on a two-step process. First, the parameters of hidden layer are adjusted by a self-organized learning algorithm. Next, the weights matrix of the output layer is determined using the Moore–Penrose inverse method. Nine classification datasets are considered to demonstrate the efficiency of the proposed approach compared with original extreme learning machine, Tikhonov regularization optimally pruned extreme learning machine, and backpropagation algorithms. The results show that the proposed method is fast and produces better accuracy and generalization performances.

The capacity estimation and cycle life prediction of lithium-ion batteries using a new broad extreme learning machine approach

Lithium-ion batteries have become the main power source of many electronic devices. Accurate capacity estimation and cycle life prediction of lithium-ion batteries are of great significance to ensure the reliability of electronic devices. Extreme learning machine(ELM) is a kind of single hidden layer feedforward neural network with fast learning speed and good generalization performance. Considering the shortcomings of deep learning and the increasing size of battery datasets, this paper introduces the idea of Broad Learning(BL) and develops a new ELM model: Broad Learning-Extreme Learning Machine(BL-ELM). First, an ELM network is constructed, and the feature nodes are produced by feature mapping of the input data. Second, the enhancement operation is performed on the mapped features to produce the enhancement nodes. Next, all these two kinds of nodes are merged to become the new input layer of the network, so that the model can quickly and fully obtain effective feature information from the input data. Finally, experiments are performed with different battery datasets. The results show that BL-ELM method can not only ensure the accuracy of estimation and prediction but also save time greatly. Further comparisons with other algorithms show that this novel model is more effective and competitive.

Densely Connected Deep Extreme Learning Machine Algorithm

As a single hidden layer feed-forward neural network, the extreme learning machine (ELM) has been extensively studied for its short training time and good generalization ability. Recently, with the deep learning algorithm becoming a research hotspot, some deep extreme learning machine algorithms such as multi-layer extreme learning machine (ML-ELM) and hierarchical extreme learning machine (H-ELM) have also been proposed. However, the deep ELM algorithm also has many shortcomings: (1) when the number of model layers is shallow, the random feature mapping makes the sample features cannot be fully learned and utilized; (2) when the number of model layers is deep, the validity of the sample features will decrease after continuous abstraction and generalization. In order to solve the above problems, this paper proposes a densely connected deep ELM algorithm: dense-HELM (D-HELM). Benchmark data sets of different sizes have been employed for the property of the D-HELM algorithm. Compared with the H-ELM algorithm on the benchmark dataset, the average test accuracy is increased by 5.34% and the average training time is decreased by 21.15%. On the NORB dataset, the proposed D-HELM algorithm still maintains the best classification results and the fastest training speed. The D-HELM algorithm can make full use of the features of hidden layer learning by using the densely connected network structure and effectively reduce the number of parameters. Compared with the H-ELM algorithm, the D-HELM algorithm significantly improves the recognition accuracy and accelerates the training speed of the algorithm.

ELM speaker identification for limited dataset using multitaper based MFCC and PNCC features with fusion score

In current scenario, speaker recognition under noisy condition is the major challenging task in the area of speech processing. Due to noise environment there is a significant degradation in the system performance. The major aim of the proposed work is to identify the speaker’s under clean and noise background using limited dataset. In this paper, we proposed a multitaper based Mel frequency cepstral coefficients (MFCC) and power normalization cepstral coefficients (PNCC) techniques with fusion strategies. Here, we used MFCC and PNCC techniques with different multitapers to extract the desired features from the obtained speech samples. Then, cepstral mean and variance normalization (CMVN) and Feature warping (FW) are the two techniques applied to normalize the obtained features from both the techniques. Furthermore, as a system model low dimension i-vector model is used and also different fusion score strategies like mean, maximum, weighted sum, cumulative and concatenated fusion techniques are utilized. Finally extreme learning machine (ELM) is used for classification in order to increase the system identification accuracy (SIA) intern which is having a single layer feedforward neural network with less complexity and time consuming compared to other neural networks. TIMIT and SITW 2016 are the two different databases are used to evaluate the proposed system under limited data of these databases. Both clean and noisy backgrounds conditions are used to check the SIA.

Solving two-dimensional linear partial differential equations based on Chebyshev neural network with extreme learning machine algorithm

PurposeThis paper aims to develop a novel algorithm and apply it to solve two-dimensional linear partial differential equations (PDEs). The proposed method is based on Chebyshev neural network and extreme learning machine (ELM) called Chebyshev extreme learning machine (Ch-ELM) method.Design/methodology/approachThe network used in the proposed method is a single hidden layer feedforward neural network. The Kronecker product of two Chebyshev polynomials is used as basis function. The weights from the input layer to the hidden layer are fixed value 1. The weights from the hidden layer to the output layer can be obtained by using ELM algorithm to solve the linear equations established by PDEs and its definite conditions.FindingsTo verify the effectiveness of the proposed method, two-dimensional linear PDEs are selected and its numerical solutions are obtained by using the proposed method. The effectiveness of the proposed method is illustrated by comparing with the analytical solutions, and its superiority is illustrated by comparing with other existing algorithms.Originality/valueCh-ELM algorithm for solving two-dimensional linear PDEs is proposed. The algorithm has fast execution speed and high numerical accuracy.

Cortico-Hippocampal Computational Modeling Using Quantum Neural Networks to Simulate Classical Conditioning Paradigms.

Most existing cortico-hippocampal computational models use different artificial neural network topologies. These conventional approaches, which simulate various biological paradigms, can get slow training and inadequate conditioned responses for two reasons: increases in the number of conditioned stimuli and in the complexity of the simulated biological paradigms in different phases. In this paper, a cortico-hippocampal computational quantum (CHCQ) model is proposed for modeling intact and lesioned systems. The CHCQ model is the first computational model that uses the quantum neural networks for simulating the biological paradigms. The model consists of two entangled quantum neural networks: an adaptive single-layer feedforward quantum neural network and an autoencoder quantum neural network. The CHCQ model adaptively updates all the weights of its quantum neural networks using quantum instar, outstar, and Widrow–Hoff learning algorithms. Our model successfully simulated several biological processes and maintained the output-conditioned responses quickly and efficiently. Moreover, the results were consistent with prior biological studies.

On Sharpness of Error Bounds for Univariate Approximation by Single Hidden Layer Feedforward Neural Networks

A new non-linear variant of a quantitative extension of the uniform boundedness principle is used to show sharpness of error bounds for univariate approximation by sums of sigmoid and ReLU functions. Single hidden layer feedforward neural networks with one input node perform such operations. Errors of best approximation can be expressed using moduli of smoothness of the function to be approximated (i.e., to be learned). In this context, the quantitative extension of the uniform boundedness principle indeed allows to construct counterexamples that show approximation rates to be best possible. Approximation errors do not belong to the little-o class of given bounds. By choosing piecewise linear activation functions, the discussed problem becomes free knot spline approximation. Results of the present paper also hold for non-polynomial (and not piecewise defined) activation functions like inverse tangent. Based on Vapnik–Chervonenkis dimension, first results are shown for the logistic function.

Gauss–Seidel Extreme Learning Machines

Extreme learning machines (ELM) were created to simplify the training phase of single-layer feedforward neural networks, where the input weights are randomly set and the only parameter is the number of neurons in the hidden layer. These networks are also known for one-shot training using Moore–Penrose pseudo-inverse. In this work, we propose Gauss–Seidel extreme learning machine (GS-ELM), an ELM based on Gauss–Seidel iterative method to solve linear equation systems. We performed tests considering databases with different characteristics and analysed its discrimination capabilities and memory consumption in comparison to the canonical ELM and the online sequential ELM. GS-ELM presented similar discrimination capabilities, but consuming significantly less memory, turning possible its application in low-memory systems and embedded solutions.

Conditioning optimization of extreme learning machine by multitask beetle antennae swarm algorithm

Extreme learning machine (ELM) as a simple and rapid neural network has been shown its good performance in various areas. Different from the general single hidden layer feedforward neural network (SLFN), the input weights and biases in hidden layer of ELM are generated randomly, so that it only takes a little computational overhead to train the model. However, the strategy of selecting input weights and biases at random may result in ill-conditioned problems. Aiming to optimize the conditioning of ELM, we propose an effective particle swarm heuristic algorithm called Multitask Beetle Antennae Swarm Algorithm (MBAS), which is inspired by the structures of artificial bee colony (ABC) algorithm and Beetle Antennae Search (BAS) algorithm. Then, the proposed MBAS is applied for optimizing the input weights and biases of ELM to solve its ill-conditioned problems. Experiment results show that the proposed method is capable of simultaneously reducing the condition number and regression error, and achieving good generalization performance.

Aircraft parameter estimation using ELM network

Purpose The purpose of this paper is to investigate the estimation methodology with a highly generalized cost-effective single hidden layer neural network. Design/methodology/approach The aerodynamic parameter estimation is a challenging research area of aircraft system identification, which finds various applications such as flight control law design and flight simulators. With the availability of the large database, the data-driven methods have gained attention, which is primarily based on the nonlinear function approximation using artificial neural networks. A novel single hidden layer feed-forward neural network (FFNN) known as extreme learning machine (ELM), which overcomes the issues such as learning rate, number of epochs, local minima, generalization performance and computational cost, as encountered in the conventional gradient learning-based FFNN has been used for the nonlinear modeling of the aerodynamic forces and moments. A mathematical formulation based on the partial differentiation is proposed to estimate the aerodynamic parameters. Findings The real flight data of longitudinal and lateral-directional motion have been considered to estimate their respective aerodynamic parameters using the proposed methodology. The efficacy of the estimates is verified with the results obtained through the conventional parameter estimation methods such as the equation-error method and filter-error method. Originality/value The present study is an outcome of the research conducted on ELM for the estimation of aerodynamic parameters from the real flight data. The proposed method is capable to estimate the parameters in the presence of noise.

Fractal-Like Kinetics of Adsorption Applied to the Solid/Solution Interface.

In this paper, the fractal-like multiexponential (f-mexp) equation was modified by introducing the fractional fractal exponent to each stage of the adsorption process. The new equation was used for the analysis of kinetic adsorption of copper onto treated attapulgite. The modeling results show that the modified f-mexp equation fits properly the kinetic data in comparison with the classical and fractal-like kinetic models tested. The effect of varying the initial concentration of the adsorbate on the kinetic parameters was analyzed. Artificial neural networks were applied for the prediction of adsorption efficiency. Outcomes indicate that the multilayer perceptron neural network can predict the removal of copper from aqueous solutions more accurately under different experimental conditions than the single-layer feedforward neural network. Single-site and multisite occupancy adsorption models were used for the analysis of experimental adsorption equilibrium data of copper onto treated attapulgite. The modeling results show that there is no multisite occupancy effect and that the equilibrium data fit well the Langmuir–Freundlich isotherm.

Monthly runoff forecasting via an improved extreme learning machine

Generally, monthly runoff prediction is of great importance for effective water resource planning and management. Extreme learning machine (ELM) is a novel training tool for the famous single layer feed-forward neural network. Due to the satisfying performance, ELM is chosen for monthly runoff forecasting in this research. Nevertheless, it is unfortunately found that ELM easily falls into local optima in practice because the randomly-determined computational parameters remain unchanged during the learning process. Thus, this paper tries to develop an improved extreme learning algorithm (IELM) where the evolutionary algorithm is used to search for satisfying computational parameters while the Moore-Penrose generalized inverse method is used to determine the output weights. The IELM method is applied to forecast the monthly runoff of Hongjiadu reservoir in southwest China. The results show that the proposed method outperforms several traditional algorithms with respect to the performance indicators. Thus, this paper provides a new and effective artificial intelligence approach for the monthly runoff forecasting.

Extreme learning with chemical reaction optimization for stock volatility prediction

Extreme learning machine (ELM) allows for fast learning and better generalization performance than conventional gradient-based learning. However, the possible inclusion of non-optimal weight and bias due to random selection and the need for more hidden neurons adversely influence network usability. Further, choosing the optimal number of hidden nodes for a network usually requires intensive human intervention, which may lead to an ill-conditioned situation. In this context, chemical reaction optimization (CRO) is a meta-heuristic paradigm with increased success in a large number of application areas. It is characterized by faster convergence capability and requires fewer tunable parameters. This study develops a learning framework combining the advantages of ELM and CRO, called extreme learning with chemical reaction optimization (ELCRO). ELCRO simultaneously optimizes the weight and bias vector and number of hidden neurons of a single layer feed-forward neural network without compromising prediction accuracy. We evaluate its performance by predicting the daily volatility and closing prices of BSE indices. Additionally, its performance is compared with three other similarly developed models—ELM based on particle swarm optimization, genetic algorithm, and gradient descent—and find the performance of the proposed algorithm superior. Wilcoxon signed-rank and Diebold–Mariano tests are then conducted to verify the statistical significance of the proposed model. Hence, this model can be used as a promising tool for financial forecasting.