Optimal Number Of Hidden Nodes Research Articles

Development of an IRMO-BPNN Based Single Pile Ultimate Axial Bearing Capacity Prediction Model

The ultimate axial bearing capacity (UABC) of a single pile is an important parameter in pile design. BP neural network (BPNN) has a strong nonlinear mapping ability and can effectively predict the UABC of a single pile. However, frequent immersion in unstable search results with local vibration leads BPNN to a less usable solution. The weights and biases of the BPNN model are optimized using the improved radial movement optimization (IRMO) algorithm in this study, and a new method named the IRMO-BP neural network (IRMO-BPNN) is proposed to predict the UABC of a single pile. The IRMO-BPNN model was developed from a database of 196 static load test (SLT) samples, and model hyper-parameter analysis was carried out to determine the optimal number of hidden nodes, population size, and the number of iterations. The prediction accuracy and stability of the IRMO-BPNN model are verified by comparing it with the GA-based ANN model, ANFIS-GMDH-PSO model, and RBFANN model. The results show that the IRMO-BPNN model can accurately predict the UABC of a single pile and improves the situation that the BPNN model is easy to fall into local optimal values and its search results are unstable. The IRMO-BPNN model has significant advantages over other models.

An optimized wavelet neural networks using cuckoo search algorithm for function approximation and chaotic time series prediction

Although the practicability of using wavelet neural networks (WNNs) in nonlinear function approximation has been addressed extensively, selecting the optimal number of hidden nodes and their appropriate initial locations remains a great challenge for WNNs’ initialization. The cuckoo search algorithm (CSA) is used in this study for optimizing WNNs. The position of the cuckoo eggs represents the translation of the wavelet hidden nodes, which are optimized based on the egg-laying and breeding strategy of cuckoos. The solutions from the CSA are assigned as initial translation vectors for the WNNs and subsequently evaluated on a few benchmarking functions and real-world applications. Performance assessment demonstrates its superior approximation capability than the existing methods used for WNNs initialization.

A fast adaptive SVDIncExtreme learning machine framework for remote sensing image classification

ABSTRACT Quick and efficient classification of images is important in many Remote Sensing Image (RSI) understanding tasks. The enormous availability of the RSI makes the classification a challenging task, which further needs to design more efficient models. Realizing the wide range of applications of the research area, a new fast Remote Sensing Image Classification (RSIC) framework using an Adaptive Singular Value Decomposition incorporated Extreme Learning Machine (ASVDIncELM) is proposed in this work. The classification framework uses Multi-Scale, Multi-Band (MSMB) Gabor filters to extract Opponent Colour Texture (OCT) features to represent the knowledge patterns. As the input knowledge patterns are High Dimensional (HD), an Adaptive Growing-Pruning Hidden Node Selection (AGPHNS) criterion is integrated with ASVDIncELM. A user-defined heuristic is used to select the optimal number of hidden nodes. The ASVDIncELM framework supports online learning of samples that come as blocks of fixed length. The efficiency of the framework is evaluated using three benchmark RSI datasets- UC Merced (UCM), RS-19, and PatternNet. The results obtained are promising for both offline and online scenarios in terms of Overall Accuracy (OA), Weighted Average (WA) of Precision, Recall, F1-Score, and training time.

Extreme Learning Machine Design for Dealing with Unrepresentative Features.

Extreme Learning Machines (ELMs) have become a popular tool for the classification of electroencephalography (EEG) signals for Brain Computer Interfaces. This is so mainly due to their very high training speed and generalization capabilities. Another important advantage is that they have only one hyperparameter that must be calibrated: the number of hidden nodes. While most traditional approaches dictate that this parameter should be chosen smaller than the number of available training examples, in this article we argue that, in the case of problems in which the data contain unrepresentative features, such as in EEG classification problems, it is beneficial to choose a much larger number of hidden nodes. We characterize this phenomenon, explain why this happens and exhibit several concrete examples to illustrate how ELMs behave. Furthermore, as searching for the optimal number of hidden nodes could be time consuming in enlarged ELMs, we propose a new training scheme, including a novel pruning method. This scheme provides an efficient way of finding the optimal number of nodes, making ELMs more suitable for dealing with real time EEG classification problems. Experimental results using synthetic data and real EEG data show a major improvement in the training time with respect to most traditional and state of the art ELM approaches, without jeopardising classification performance and resulting in more compact networks.

Classification of Bidikmisi Scholarship Acceptance using Neural Network Based on Hybrid Method of Genetic Algorithm

A Neural network is a series of algorithms that endeavours to recognize underlying relationships in a set of data through processes that mimic the way human brains operate. In the case of classification, this method can provide a fit model through various factors, such as the variety of the optimal number of hidden nodes, the variety of relevant input variables, and the selection of optimal connection weights. One popular method to achieve the optimal selection of connection weights is using a Genetic Algorithm (GA), the basic concept is to iterate over Darwin's evolution. This research presents the Neural Network method with the Backpropagation Neural Network (BPNN) and the combined method of BPNN with GA, where GA is used to initialize and optimize the connection weight of BPNN. Based on accuracy value, the BPNN method combined with GA provides better classification, which is 90.51%, in the case of Bidikmisi Scholarship classification in East Java.

Pruning Multilayered ELM Using Cholesky Factorization and Givens Rotation Transformation

Extreme learning machine is originally proposed for the learning of the single hidden layer feedforward neural network to overcome the challenges faced by the backpropagation (BP) learning algorithm and its variants. Recent studies show that ELM can be extended to the multilayered feedforward neural network in which the hidden node could be a subnetwork of nodes or a combination of other hidden nodes. Although the ELM algorithm with multiple hidden layers shows stronger nonlinear expression ability and stability in both theoretical and experimental results than the ELM algorithm with the single hidden layer, with the deepening of the network structure, the problem of parameter optimization is also highlighted, which usually requires more time for model selection and increases the computational complexity. This paper uses Cholesky factorization strategy and Givens rotation transformation to choose the hidden nodes of MELM and obtains the number of nodes more suitable for the network. First, the initial network has a large number of hidden nodes and then uses the idea of ridge regression to prune the nodes. Finally, a complete neural network can be obtained. Therefore, the ELM algorithm eliminates the need to manually set nodes and achieves complete automation. By using information from the previous generation’s connection weight matrix, it can be evitable to re-calculate the weight matrix in the network simplification process. As in the matrix factorization methods, the Cholesky factorization factor is calculated by Givens rotation transform to achieve the fast decreasing update of the current connection weight matrix, thus ensuring the numerical stability and high efficiency of the pruning process. Empirical studies on several commonly used classification benchmark problems and the real datasets collected from coal industry show that compared with the traditional ELM algorithm, the pruning multilayered ELM algorithm proposed in this paper can find the optimal number of hidden nodes automatically and has better generalization performance.

Optimal Design and Feature Selection by Genetic Algorithm for Emotional Artificial Neural Network (EANN) in Rainfall-Runoff Modeling

Rainfall-runoff (r-r) modeling at different time scales is considered as a significant issue in hydro-environmental planning. As a first hydrological implementation, for one-time-ahead r-r modeling of two watersheds with totally distinct climatic conditions, Genetic Algorithm (GA, as a global search technique) and Emotional Artificial Neural Network (EANN, as a new production of Artificial Intelligence (AI) based methods that simulated based on the brain neurophysiological structure) was combined. Determining the optimal architecture of AI-based networks is vital for increasing the accuracy of prediction by the network and also to reduce run-time. In the current study, GA has been implemented to choose the important features candidate as EANN input and automatically diagnose the optimal number of hidden nodes and hormones simultaneously. The acquired results indicated a better representation of the proposed hybrid GA-EANN model compared to the sole ANN and EANN. Numerical identification of obtained results revealed that the proposed hybrid GA-EANN model might enhance the better results than the EANN model up to 19% and 35% in terms of testing suitability criteria for Aji Chai and Murrumbidgee catchments, respectively.

An Enhanced Unsupervised Extreme Learning Machine Based Method for the Nonlinear Fault Detection

Although the unsupervised extreme learning machine (UELM) based methods have been widely used to diagnosis the nonlinear process faults recently, the UELM algorithm is only designed to preserve the local adjacency similarity of the input dataset instead of mining the intra-class variations. Besides, the determination of the optimal UELM hidden nodes number is a tough issue. In order to deal with these two problems, a novel enhanced UELM (EUELM) based scheme is developed to effectively detect the nonlinear process faults in our work. In the proposed EUELM approach, the UELM algorithm is first improved by naturally incorporating the diversity analysis technique into the original UELM objective function to preserve both the intra-class variation and the local adjacency similarity of the input dataset. Then, to settle the difficult issue of selecting the optimal number of hidden nodes, kernel trick is further employed in the EUELM approach to mine the data strong nonlinearity. Based on the extracted diversity and local similarity low dimensional feature information, the k-nearest neighbor (KNN) principle is applied to derive a monitoring statistic for fault detection. At last, the experiments and comparisons on the monitoring effectiveness of the suggested EUELM based approach are made on a numerical nonlinear system and the benchmark Tennessee Eastman (TE) process. The obtained monitoring results illustrate that the significant improvements can be achieved by the proposed EUELM based fault detection approach compared with other popular and related approaches.

Development of artificial neural networks for an energy storage system generated power prediction

As energy utility firms expand their production outputs from renewable energy resources, interest in investment in an energy storage system (ESS) will increase in the coming years. This article determines the optimal number of hidden nodes and the maximum number of epochs to generate the best prediction results for this generated power estimation. Primary energy efficiency, energy ratio, and ambient air temperature were used to predict the generated power of an ESS. Experimental data sets for compressed air energy storage (CAES) were used to train a single hidden layer artificial neural network (ANN) model. The developed ANN models for CAES and compressed air storage with humidification (CASH) show quick convergence to the targeted error function. Based on eight different artificial neural network architectures, a 3−5−1 ANN architecture generated the best results. This architecture mean squared error was 0.00042. Using experimental data for CASH to validate the single hidden layer model, the ANN model performed satisfactorily.

A novel algorithm of Nested-ELM for predicting blasting vibration

The prediction model of blasting vibration has always been a hot and difficult topic because of the very complex nonlinear relationship between the blasting vibration and its influencing factors. A novel algorithm of Nested-ELM for predicting blasting vibration was proposed in this paper. Nested-ELM algorithm can quickly select the optimal input weights and biases of hidden nodes by setting MSE as the fitness function and combining with RWS method. And the algorithm can also quickly determine the optimal number of hidden nodes by setting its initial value according to the empirical formulas and selecting MAPE as the diffusion search index. The feasibility and superiority of Nested-ELM algorithm for predicting blasting vibration were proved by the application of Nested-ELM model on four different types of blasting vibration samples. This paper can provide a novel improved ELM algorithm for predicting blasting vibration with good performance in operation efficiency, prediction accuracy, generalization and sample-number independence.

A Novel Method of Curve Fitting Based on Optimized Extreme Learning Machine

ABSTRACT In this article, we present a new method based on extreme learning machine (ELM) algorithm for solving nonlinear curve fitting problems. Curve fitting is a computational problem in which we seek an underlying target function with a set of data points given. We proposed that the unknown target function is realized by an ELM with introducing an additional linear neuron to correct the localized behavior caused by Gaussian type neurons. The number of hidden layer neurons of ELM is a crucial factor to achieve a good performance. An evolutionary computation algorithm–particle swarm optimization (PSO) technique is applied to determine the optimal number of hidden nodes. Several numerical experiments with benchmark datasets, simulated spectral data and measured data from high energy physics experiments have been conducted to test the proposed method. Accurate fitting has been accomplished for various tough curve fitting tasks. Comparing with the results of other methods, the proposed method outperforms the traditional numerical-based technique. This work clearly demonstrates that the classical numerical analysis problem-curve fitting can be satisfactorily resolved via the approach of artificial intelligence.

Extreme learning with chemical reaction optimization for stock volatility prediction

Extreme learning machine (ELM) allows for fast learning and better generalization performance than conventional gradient-based learning. However, the possible inclusion of non-optimal weight and bias due to random selection and the need for more hidden neurons adversely influence network usability. Further, choosing the optimal number of hidden nodes for a network usually requires intensive human intervention, which may lead to an ill-conditioned situation. In this context, chemical reaction optimization (CRO) is a meta-heuristic paradigm with increased success in a large number of application areas. It is characterized by faster convergence capability and requires fewer tunable parameters. This study develops a learning framework combining the advantages of ELM and CRO, called extreme learning with chemical reaction optimization (ELCRO). ELCRO simultaneously optimizes the weight and bias vector and number of hidden neurons of a single layer feed-forward neural network without compromising prediction accuracy. We evaluate its performance by predicting the daily volatility and closing prices of BSE indices. Additionally, its performance is compared with three other similarly developed models—ELM based on particle swarm optimization, genetic algorithm, and gradient descent—and find the performance of the proposed algorithm superior. Wilcoxon signed-rank and Diebold–Mariano tests are then conducted to verify the statistical significance of the proposed model. Hence, this model can be used as a promising tool for financial forecasting.

Intelligent approach for large-scale data mining

Large-scale data mining has become a very difficult issue using traditional methods because the data complexity is very high. In the proposed approach, an integration of three methods; Optimised Principal Component Analysis (OPCA), Optimised Enhanced Extreme Learning Machine (OEELM), and stratified sampling, called OPCA-EELM2SS, is presented to provide intelligent and enhanced large-scale data mining. OPCA provides a good representation of large-scale data sets by using the Stratified Sample (SS). By using OEELM, the optimal number of Hidden Nodes (HNs) in ELM is exploited to build a single hidden layer feedforward neural network (SLFN). The proposed approach is tested by using nineteen benchmark data sets. The experimental results demonstrate the effectiveness of the proposed approach by performing different experiments for classical PCA and Independent Component Analysis (ICA), which are integrated with the enhanced ELM using different evaluation criteria. For more reliability, the proposed approach is compared with many previous methods.

Laplacian least learning machine with dynamic updating for imbalanced classification

When handling imbalanced datasets, the existing least learning machine ignores both the interrelationship between instances and the prior knowledge about the classes and hence tends to provide unfavorable accuracies across the classes. With the Laplacian matrix based loss function, Laplacian least learning machine (L2MM) is developed to address the infeasibility of the existing least learning machine for imbalanced data classification in this study. In order to find out the optimal number of hidden nodes in L2MM, by means of the derived update equations without any re-calculation of inverse matrix, its two incremental versions (i.e., with and without regularization) are invented for dynamically updating the hidden nodes in one-by-one way, thereby saving much training time. L2MM and its incremental versions inherit fast learning and good generalization capability of least learning machine. Experimental results on 19 benchmarking imbalanced datasets indicate the effectiveness of the proposed incremental L2MM for imbalanced classification tasks.

Learning Optimized Structure of Neural Networks by Hidden Node Pruning With L1 Regularization.

We propose three different methods to determine the optimal number of hidden nodes based on L1 regularization for a multilayer perceptron network. The first two methods, respectively, use a set of multiplier functions and multipliers for the hidden-layer nodes and implement the L1 regularization on those, while the third method equipped with the same multipliers uses a smoothing approximation of the L1 regularization. Each of these methods begins with a given number of hidden nodes, then the network is trained to obtain an optimal architecture discarding redundant hidden nodes using the multiplier functions or multipliers. A simple and generic method, namely, the matrix-based convergence proving method (MCPM), is introduced to prove the weak and strong convergence of the presented smoothing algorithms. The performance of the three pruning methods has been tested on 11 different classification datasets. The results demonstrate the efficient pruning abilities and competitive generalization by the proposed methods. The theoretical results are also validated by the results.

Rolling Force Prediction of Hot Rolling Based on GA-MELM

In the hot continuous rolling process, the main factor affecting the actual thickness of strip is the rolling force. The precision of rolling force calculation is the key to realize accurate on-line control. However, because of the complexity and nonlinearity of the rolling process, as well as many influencing factors, the theoretical analysis of the traditional rolling force prediction model often needs to be simplified and hypothesized. This leads to the incompleteness of the mathematical model and the deviation between the calculated results and the actual working conditions. In this paper, a rolling force prediction method based on genetic algorithm (GA), particle swarm optimization algorithm (PSO), and multiple hidden layer extreme learning machine (MELM) is proposed, namely, PSO-GA-MELM algorithm, which takes MELM as the basic model for rolling force prediction. In the modeling process, GA is used to determine the optimal number of hidden layers and the optimal number of hidden nodes, and PSO is used to search for the optimal input weights and biases. This method avoids the influence of human intervention on the model and saves the modeling time. This paper takes the actual production data of BaoSteel 2050 production line as experimental data, and the experimental results indicate that the algorithm can be effectively used to determine the optimal network structure of MELM. The rolling force prediction model trained by the algorithm has excellent performance in prediction accuracy, computational stability, and the number of hidden nodes and is applicable to the prediction of rolling force in hot continuous rolling process.

Optimizing wavelet neural networks using modified cuckoo search for multi-step ahead chaotic time series prediction

Determining the optimal number of hidden nodes and their proper initial locations are essentially crucial before the wavelet neural networks (WNNs) start their learning process. In this paper, a novel strategy known as the modified cuckoo search algorithm (MCSA), is proposed for WNNs initialization in order to improve its generalization performance. The MCSA begins with an initial population of cuckoo eggs, which represent the translation vectors of the wavelet hidden nodes, and subsequently refines their locations by imitating the breeding mechanism of cuckoos. The resulting solutions from the MCSA are then used as the initial translation vectors for the WNNs. The feasibility of the proposed method is evaluated by forecasting a benchmark chaotic time series, and its superior prediction accuracy compared with that of conventional WNNs demonstrates its potential benefit.

Adaptive neural network control with optimal number of hidden nodes for trajectory tracking of robot manipulators

In this paper, an adaptive neural network control with optimal number of hidden nodes and less computation is proposed for approximating the system uncertainty and tracking the trajectory of robot manipulators. Unlike the existing researches on adaptive neural network for robot manipulators, whose number of hidden nodes is fixed and determined through the trial and error, a new approach is proposed to obtain the optimal number of hidden nodes, in which the number of hidden nodes adapts to the trajectory variations and is capable of catching up with the optimal value and minimizing the tracking error. The proposed control scheme can avoid overfitting and underfitting problems and guarantee a better trajectory tracking. Mathematical proof for stability and convergence of the system is presented using Lyapunov method. In the end, simulations are performed to illustrate the effectiveness of the proposed approach.

Multivariate Chaotic Time Series Prediction Based on ELM–PLSR and Hybrid Variable Selection Algorithm

In this paper, a novel method (Hybrid–ELM–PLSR) is proposed based on hybrid variable selection algorithm and improved extreme learning machine (ELM) for multivariate chaotic time series prediction. The hybrid variable selection algorithm combines the advantages of filter and wrapper, effectively balancing the calculation speed and prediction accuracy. Moreover, for ELM, multicollinearity, which can result in ill-condition, is always existent among the hidden layer output matrix. And the optimal number of hidden nodes is also difficult to be determined. Therefore,in order to overcome these problems, an improved ELM (ELM–PLSR) is proposed based on partial least square regression (PLSR). It can effectively enhance the stability performance and prediction performance of ELM. Hybrid–ELM–PLSR can be divided into three stages. At first, filter is used to rearrange the input variables through the correlations with desired variables. Then wrapper is used to select the optimal variable subset through evaluating the prediction performance of different subsets. Finally, ELM–PLSR is used to build the prediction model. The simulation experiment results based on San Francisco river runoff dataset demonstrate that the proposed method is effective for multivariate chaotic time series. And the prediction accuracy and reliability are higher than other methods.

Weighted recalibration of the Rosetta pedotransfer model with improved estimates of hydraulic parameter distributions and summary statistics (Rosetta3)

Pedotransfer functions (PTFs) have been widely used to predict soil hydraulic parameters in favor of expensive laboratory or field measurements. Rosetta (Schaap et al., 2001, denoted as Rosetta1) is one of many PTFs and is based on artificial neural network (ANN) analysis coupled with the bootstrap re-sampling method which allows the estimation of van Genuchten water retention parameters (van Genuchten, 1980, abbreviated here as VG), saturated hydraulic conductivity (Ks), and their uncertainties. In this study, we present an improved set of hierarchical pedotransfer functions (Rosetta3) that unify the water retention and Ks submodels into one. Parameter uncertainty of the fit of the VG curve to the original retention data is used in the ANN calibration procedure to reduce bias of parameters predicted by the new PTF. One thousand bootstrap replicas were used to calibrate the new models compared to 60 or 100 in Rosetta1, thus allowing the uni-variate and bi-variate probability distributions of predicted parameters to be quantified in greater detail. We determined the optimal weights for VG parameters and Ks, the optimal number of hidden nodes in ANN, and the number of bootstrap replicas required for statistically stable estimates. Results show that matric potential-dependent bias was reduced significantly while root mean square error (RMSE) for water content were reduced modestly; RMSE for Ks was increased by 0.9% (H3w) to 3.3% (H5w) in the new models on log scale of Ks compared with the Rosetta1 model. It was found that estimated distributions of parameters were mildly non-Gaussian and could instead be described rather well with heavy-tailed α-stable distributions. On the other hand, arithmetic means had only a small estimation bias for most textures when compared with the mean-like “shift” parameter of the α-stable distributions. Arithmetic means and (co-)variances are therefore still recommended as summary statistics of the estimated distributions. However, it may be necessary to parameterize the distributions in different ways if the new estimates are used in stochastic analyses of vadose zone flow and transport. Rosetta1 and Posetta3 were implemented in the python programming language, and the source code as well as additional documentation is available at: http://www.cals.arizona.edu/research/rosettav3.html.