Optimal Number Of Neurons Research Articles

An Internet of Things System for Underground Mine Air Quality Pollutant Prediction Based on Azure Machine Learning.

The implementation of wireless sensor networks (WSNs) for monitoring the complex, dynamic, and harsh environment of underground coal mines (UCMs) is sought around the world to enhance safety. However, previously developed smart systems are limited to monitoring or, in a few cases, can report events. Therefore, this study introduces a reliable, efficient, and cost-effective internet of things (IoT) system for air quality monitoring with newly added features of assessment and pollutant prediction. This system is comprised of sensor modules, communication protocols, and a base station, running Azure Machine Learning (AML) Studio over it. Arduino-based sensor modules with eight different parameters were installed at separate locations of an operational UCM. Based on the sensed data, the proposed system assesses mine air quality in terms of the mine environment index (MEI). Principal component analysis (PCA) identified CH4, CO, SO2, and H2S as the most influencing gases significantly affecting mine air quality. The results of PCA were fed into the ANN model in AML studio, which enabled the prediction of MEI. An optimum number of neurons were determined for both actual input and PCA-based input parameters. The results showed a better performance of the PCA-based ANN for MEI prediction, with R2 and RMSE values of 0.6654 and 0.2104, respectively. Therefore, the proposed Arduino and AML-based system enhances mine environmental safety by quickly assessing and predicting mine air quality.

A non-iterative method for pruning hidden neurons in neural networks with random weights

Neural networks with random weights have the advantage of fast computational time in both training and testing. However, one of the main challenges of single layer feedforward neural networks is the selection of the optimal number of neurons in the hidden layer, since few/many neurons lead to problems of underfitting/overfitting. Adapting Garson's algorithm, this paper introduces a new efficient and fast non-iterative algorithm for the selection of neurons in the hidden layer for randomization based neural networks. The proposed approach is divided into three steps: (1) train the network with h hidden neurons, (2) apply Garson's algorithm to the matrix of the hidden layer, and (3) perform pruning reducing hidden layer neurons based on the harmonic mean. Our experiments in regression and classification problems confirmed that the combination of the pruning technique with these types of neural networks improved their predictive performance in terms of mean square error and accuracy. Additionally, we tested our proposed pruning method with neural networks trained under sequential learning algorithms, where Random Vector Functional Link obtained, in general, the best predictive performance compared to online sequential versions of extreme learning machines and single hidden layer neural network with random weights.

Mathematical and artificial neural network modeling of production of ethylene from ethane pyrolysis in a tubular reactor

ABSTRACTPyrolysis of lower alkanes is among the main industrial methods for the production of light olefins. In this paper, an artificial neural network (ANN) model was developed in order to predict production of ethylene from ethane pyrolysis based on data obtained from mathematical modeling of the process in a plug flow reactor. Results obtained from the mathematical models were validated using experimental data from the literature. According to the results, the linear regression between network outputs and corresponding targets are proven satisfactory with a correlation coefficient of 1. The optimum number of neurons of 10 was obtained at hidden layer.

Non-intrusive reduced order modeling of nonlinear problems using neural networks

We develop a non-intrusive reduced basis (RB) method for parametrized steady-state partial differential equations (PDEs). The method extracts a reduced basis from a collection of high-fidelity solutions via a proper orthogonal decomposition (POD) and employs artificial neural networks (ANNs), particularly multi-layer perceptrons (MLPs), to accurately approximate the coefficients of the reduced model. The search for the optimal number of neurons and the minimum amount of training samples to avoid overfitting is carried out in the offline phase through an automatic routine, relying upon a joint use of the Latin hypercube sampling (LHS) and the Levenberg–Marquardt (LM) training algorithm. This guarantees a complete offline-online decoupling, leading to an efficient RB method – referred to as POD-NN – suitable also for general nonlinear problems with a non-affine parametric dependence. Numerical studies are presented for the nonlinear Poisson equation and for driven cavity viscous flows, modeled through the steady incompressible Navier–Stokes equations. Both physical and geometrical parametrizations are considered. Several results confirm the accuracy of the POD-NN method and show the substantial speed-up enabled at the online stage as compared to a traditional RB strategy.

A Prediction Method for Galloping of Transmission Lines Based on Improved Neural Network

The traditional back-propagation neural network (BP) have the disadvantages including the random generation of initial weights and thresholds, easy to fall into the local optimization, and the convergence speed is slow, and it’s hard to confirm the number of neurons in hidden layer. In this paper, the Genetic Algorithm (GA) is utilized to optimize the initial weights and thresholds space of the BP neural network. To obtain the optimal weight matrix and threshold matrix, the error-forward-feedback neural network training is carried out by using the data of transmission line galloping. The trial and error method are used to reduce the number of hidden layer neurons and find the optimal number of neurons. An optimized GA-BP neural network model is established to warn the occurrence of transmission line galloping. The historical data of the transmission lines galloping in the related areas is analyzed by the optimized GA-BP neural network model. The validity and practicability of the proposed GA-BP neural network model is tested and verified. The simulation results show that the GA-BP neural network module could predict the galloping situation of transmission lines more accurately and effectively. As a result, it provides a strong guarantee for preventing large-scale grid fault disasters, and further improves the power grid's ability to withstand natural disasters.

Estimation of Non-Revenue Water Ratio Using MRA and ANN in Water Distribution Networks

The non-revenue water (NRW) ratio in water distribution networks is the ratio of losses from unbilled authorized consumption and apparent and real losses to the total water supply. NRW is an important parameter for prioritizing the improvement of a water distribution system and identifying the influencing parameters. Though the method using multiple regression analysis (MRA) is a statistical analysis method for estimating the NRW ratio using the main parameters of a water distribution system, it has disadvantages in that the accuracy is low compared to the measured NRW ratio. In this study, an artificial neural network (ANN) was applied to estimate the NRW ratio to improve assessment accuracy and suggest an efficient methodology to identify related parameters of the NRW ratio. When using an ANN with the optimal number of neurons, the accuracy of estimation was higher than that of conventional statistical methods, as with MRA.

New Approach to Optimize the Rate of Penetration Using Artificial Neural Network

Rate of penetration (ROP) is one of the most important parameters of the drilling operation. Optimizing the ROP will reduce the overall cost of the drilling process. ROP depends on many variables such as drilling parameters [flow rate (Q), RPM, torque (T), weight on bit (WOB), stand pipe pressure (P)], fluid properties (mud density and plastic viscosity), and formation strength (UCS). The objectives of this paper are (1) to evaluate the effect of the drilling parameters and the drilling fluid parameters on rate of penetration (ROP), (2) construct a new artificial neural network (ANN) technique to estimate the ROP as a function of drilling parameters and fluid properties using actual field measurements (3333 data points), and (3) converting the black box of the ANN model to a white box by developing a new ROP correlation based on the optimized weights and biases of the developed model. The optimization process of the ANN model showed that the optimum number of neurons was 20, which resulted in the lowest average absolute percentage error (AAPE) and the highest correlation coefficient (R). The developed ANN model was able to estimate ROP with high accuracy (R of 0.99 and AAPE of 5.6%). The developed empirical correlation for ROP prediction outperformed the previous models. The high accuracy of the developed correlation (AAPE of 4%) confirmed the importance of compiling the drilling parameters and the drilling fluid properties.

Two Novel Versions of Randomized Feed Forward Artificial Neural Networks: Stochastic and Pruned Stochastic

Although high accuracies were achieved by artificial neural network (ANN), determining the optimal number of neurons in the hidden layer and the activation function is still an open issue. In this paper, the applicability of assigning the number of neurons in the hidden layer and the activation function randomly was investigated. Based on the findings, two novel versions of randomized ANNs, which are stochastic, and pruned stochastic, were proposed to achieve a higher accuracy without any time-consuming optimization stage. The proposed approaches were evaluated and validated by the basic versions of the popular randomized ANNs [1] are the random weight neural network [2], the random vector functional links [3] and the extreme learning machine [4] methods. In the stochastic version of randomized ANNs, not only the weights and biases of the neurons in the hidden layer but also the number of neurons in the hidden layer and each activation function were assigned randomly. In pruned stochastic version of these methods, the winner networks were pruned according to a novel strategy in order to produce a faster response. Proposed approaches were validated via 60 datasets (30 classification and 30 regression datasets). Obtained accuracies and time usages showed that both versions of randomized ANNs can be employed for classification and regression.

Development of an intelligent system for the prognostication of energy produced by photovoltaic cells in smart grid systems

Solution of the problem of prognostication of the generated energy was proposed on the basis of mathematical apparatus of neural-fuzzy networks. The conceptual model of the household information system as a part of the common SMART GRID system was proposed. The main task of this system is continuous monitoring of the power net, prognostication of consumption of the energy consumed by domestic appliances and the energy produced by photovoltaic cell panels. Current and predicted data were obtained based on the use of current sensors and mathematical apparatus of neural-fuzzy logic. Importance and necessity of using SMART GRID technology for improving efficiency of power net operation was shown. Application of such systems can reduce energy costs and the environmental impact of energy systems. This effect is achieved by prognostication of the energy generated by domestic renewable energy sources, in particular photovoltaic cell panels which ensures more efficient energy management. Also, the proposed model of the information system makes it possible to account produced and consumed energy which enables creation of an energy-efficient operation schedule of household appliances. Analysis of the dependence of the forecast accuracy on the choice of input characteristics was made. As a result, the optimal number of neurons in the inner layer was empirically set to 250 with a prediction error within 5 %. Influence of weather factors on accuracy of the resulting forecasts was considered. In particular, it has been found that quite significant differences between actual and projected data (up to 12 %) are due to the inaccuracy of local forecasts. The proposed information model can be integrated into existing or designed systems of the Smart Home type.

Estimation of Non-Revenue Water Ratio for Sustainable Management Using Artificial Neural Network and Z-Score in Incheon, Republic of Korea

The non-revenue water (NRW) ratio in a water distribution system is the ratio of the loss due to unbilled authorized consumption, apparent losses and real losses to the overall system input volume (SIV). The method of estimating the NRW ratio by measurement might not work in an area with no district metered areas (DMAs) or with unclear administrative district. Through multiple regression analyses is a statistical analysis method for calculating the NRW ratio using the main parameters of the water distribution system, although its disadvantage is lower accuracy than that of the measured NRW ratio. In this study, an artificial neural network (ANN) was used to estimate the NRW ratio. The results of the study proved that the accuracy of NRW ratio calculated by the ANN model was higher than by multiple regression analysis. The developed ANN model was shown to have an accuracy that varies depending on the number of neurons in the hidden layer. Therefore, when using the ANN model, the optimal number of neurons must be determined. In addition, the accuracy of the outlier removal condition was higher than that of the original data used condition.

Prediction of temperature and HAZ in thermal-based processes with Gaussian heat source by a hybrid GA-ANN model

Thermal-based processes with Gaussian heat source often produce excessive temperature which can impose thermally-affected layers in specimens. Therefore, the temperature distribution and Heat Affected Zone (HAZ) of materials are two critical factors which are influenced by different process parameters. Measurement of the HAZ thickness and temperature distribution within the processes are not only difficult but also expensive. This research aims at finding a valuable knowledge on these factors by prediction of the process through a novel combinatory model. In this study, an integrated Artificial Neural Network (ANN) and genetic algorithm (GA) was used to predict the HAZ and temperature distribution of the specimens. To end this, a series of full factorial design of experiments were conducted by applying a Gaussian heat flux on Ti-6Al-4V at first, then the temperature of the specimen was measured by Infrared thermography. The HAZ width of each sample was investigated through measuring the microhardness. Secondly, the experimental data was used to create aGA-ANN model. The efficiency of GA in design and optimization of the architecture of ANN was investigated. The GA was used to determine the optimal number of neurons in hidden layer, learning rate and momentum coefficient of both output and hidden layers of ANN. Finally, the reliability of models was assessed according to the experimental results and statistical indicators. The results demonstrated that the combinatory model predicted the HAZ and temperature more effective than a trial-and-error ANN model.

Application of artificial neural networks to predict peak flow of Surma River in Sylhet Zone of Bangladesh

River flow analysis and prediction is an important task in water resources planning, particularly for a disaster-prone agricultural country like Bangladesh. The present study used two ANN models namely radial basis function (RBF) and multi-layer perceptron (MLP) to analyse Surma River flow and estimate its peak flow concentration based on five input parameters. The performances of selected models were measured using the correlation coefficient (R), mean absolute error (MAE) and model efficiency (EFF%). However, RBF network model performed better than MLP network model with high model efficiency (99.55%), low mean squared errors (38.60) and high correlation coefficient (0.996), where the optimum number of neurons was 18 for RBF and 22 for MPL network. Moreover, the proposed ANN models could be used successfully in estimating the peak-flow of the Surma River, which would facilitate water resources management policy of this region.

Application of artificial neural networks to predict peak flow of Surma River in Sylhet Zone of Bangladesh

River flow analysis and prediction is an important task in water resources planning, particularly for a disaster-prone agricultural country like Bangladesh. The present study used two ANN models namely radial basis function (RBF) and multi-layer perceptron (MLP) to analyse Surma River flow and estimate its peak flow concentration based on five input parameters. The performances of selected models were measured using the correlation coefficient (R), mean absolute error (MAE) and model efficiency (EFF%). However, RBF network model performed better than MLP network model with high model efficiency (99.55%), low mean squared errors (38.60) and high correlation coefficient (0.996), where the optimum number of neurons was 18 for RBF and 22 for MPL network. Moreover, the proposed ANN models could be used successfully in estimating the peak-flow of the Surma River, which would facilitate water resources management policy of this region.

Diabetes Classification using Radial Basis Function Network by Combining Cluster Validity Index and BAT Optimization with Novel Fitness Function

Diabetes is one of the foremost causes for the increase in mortality among children and adults in recent years. Classification systems are being used by doctors to analyse and diagnose the medical data. Radial basis function neural networks are more attractive for classification of diseases, especially in diabetes classification, because of it’s non iterative nature. Radial basis function neural networks are four layer feed forward neural network with input layer, pattern layer, summation layer and the decision layer respectively. The size of the pattern layer increases on par with training data set size. Though various attempts have made to solve this issue by clustering input data using different clustering algorithms like k-means, k-medoids, and SOFM etc. However main difficulty of determining the optimal number of neurons in the pattern layer remain unsolved. In this paper, we present a new model based on cluster validity index with radial basis neural network for classification of diabetic patients data. We employ cluster validity index in class by class fashion for determining the optimal number of neurons in pattern layer. A new convex fitness function has also been designed for bat inspired optimization algorithm to identify the weights between summation layer and pattern layer. The proposed model for radial basis function neural network is tested on Pima Indians Diabetes data set and synthetic data sets. Experimental results proved that our approach performs better in terms of accuracy, sensitivity, specificity, classification time, training time, network complexity and computational time compared to conventional radial basis function neural network. It is also proved that proposed model performs better compared to familiar classifiers namely probabilistic neural network, feed forward neural network, cascade forward network, time delay network, artificial immuine system and GINI classifier.

Diabetes-Finder: A Bat Optimized Classification System for Type-2 Diabetes

Type-2 Diabetes is one of the foremost causes for the increase in mortality across the world-wide. In this context, classification systems help doctors by analyzing the disease data. Radial Basis Function Neural Networks (RBFNN) are extensively used as classifier in medical domain because of its non-iterative nature. The size of the RBFNNs hidden-layer increases on par with dataset size. It’s difficult to determining the optimal number of neurons in hidden-layer by cost effectively. In this paper, to address this problem, we have proposed Bat-based clustering algorithm. The proposed method experimented on Pima Indians Diabetes dataset and results outperform the competing approaches.

Customer Credit Risk Assessment using Artificial Neural Networks

Since the granting of banking facilities in recent years has faced problems such as customer credit risk and affects the profitability directly, customer credit risk assessment has become imperative for banks and it is used to distinguish good applicants from those who will probably default on repayments. In credit risk assessment, a score is assigned to each customer then by comparing it with the cut-off point score which distinguishes two classes of the applicants, customers are classified into two credit statuses either a good or bad applicant. Regarding good performance and their ability of classification, generalization and learning patterns, Multilayer Perceptron Neural Network model trained using various Back-Propagation (BP) algorithms considered in designing an evaluation model in this study. The BP algorithms, Levenberg-Marquardt (LM), Gradient descent, Conjugate gradient, Resilient, BFGS Quasinewton, and One-step secant were utilized. Each of these six networks runs and trains for different numbers of neurons within their hidden layer. Mean squared error (MSE) is used as a criterion to specify optimum number of neurons in the hidden layer. The results showed that LM algorithm converges faster to the network and achieves better performance than the other algorithms. At last, by comparing classification performance of neural network with a number of classification algorithms such as Logistic Regression and Decision Tree, the neural network model outperformed the others in customer credit risk assessment. In credit models, because the cost that Type II error rate imposes to the model is too high, therefore, Receiver Operating Characteristic curve is used to find appropriate cut-off point for a model that in addition to high Accuracy, has lower Type II error rate.

Energy-efficient population coding constrains network size of a neuronal array system.

We consider the open issue of how the energy efficiency of the neural information transmission process, in a general neuronal array, constrains the network size, and how well this network size ensures the reliable transmission of neural information in a noisy environment. By direct mathematical analysis, we have obtained general solutions proving that there exists an optimal number of neurons in the network, where the average coding energy cost (defined as energy consumption divided by mutual information) per neuron passes through a global minimum for both subthreshold and superthreshold signals. With increases in background noise intensity, the optimal neuronal number decreases for subthreshold signals and increases for suprathreshold signals. The existence of an optimal number of neurons in an array network reveals a general rule for population coding that states that the neuronal number should be large enough to ensure reliable information transmission that is robust to the noisy environment but small enough to minimize energy cost.

Structure optimisation of input layer for feed-forward NARX neural network

This paper presents an optimisation method for reducing the number of input channels and the complexity of the feed-forward NARX neural network (NN) without compromising the accuracy of the NN model. By utilising the correlation analysis method, the most significant regressors are selected to form the input layer of the NN structure. Applications of vehicle handling and ride model identification are presented in this paper to demonstrate the optimisation technique, and the optimal input layer structure and the optimal number of neurons for the NN models are investigated. The results show that the developed NN model requires significantly fewer coefficients and less training time while maintaining high simulation accuracy compared with that of the unoptimised model.

Modelling and prediction of bioethanol production from intermediates and byproduct of sugar beet processing using neural networks

The aim of this work was to model and predict the process of bioethanol production from intermediates and byproduct of sugar beet processing by applying artificial neural networks. Prediction of one substrate fermentation by neural networks had the same input variables (fermentation time and starting sugar content) and one output value (ethanol content, yeast cell number or sugar content). Results showed that a good prediction model could be obtained by networks with single hidden layer. The neural network configuration that gave the best prediction for raw or thin juice fermentation was one with 8 neurons in hidden layer for all observed outputs. On the other side, the optimal number of neurons in hidden layer was found to be 9 and 10 for thick juice and molasses, respectively. Further, all substrates data were merged, which led to introducing an additional input (substrate type) and defining all outputs optimal network architecture to 3-12-1. From the results the conclusion was that artificial neural networks are a good prediction tool for the selected network outputs. Also, these predictive capabilities allowed the application of the Garson's equation for estimating the contribution of selected process parameters on the defined outputs with satisfactory accuracy.

A hybrid wavelet neural network model with mutual information and particle swarm optimization for forecasting monthly rainfall

In this paper, a hybrid wavelet neural network (HWNN) model is developed for effectively forecasting monthly rainfall from antecedent monthly rainfall and climate indices by incorporating the multiresolution analysis (MRA), mutual information (MI) and particle swarm optimization (PSO) into artificial neural network (ANN) models. The standardized monthly rainfall anomaly and large-scale climate indices are first decomposed by using the maximal overlap discrete wavelet transform (MODWT) into a certain number of subseries components with different time scales. Then at each time scale, a small and appropriate subset of ANN inputs is identified by the modified partial MI algorithm from a set of candidate subseries components with different lags for forecasting the rainfall anomaly subseries at the corresponding scale. The optimal number of neurons in the hidden layers of ANNs is determined by PSO algorithm, and then each of rainfall anomaly subseries is forecasted from the selected predictor subseries. Finally, the monthly rainfall forecast is achieved by summing all the predicted anomaly subseries and applying the inverse transform of standardized monthly rainfall. The proposed HWNN method is examined with 255 rain gauge stations over Australia, and compared to the reference methods based on the undecomposed time series. The forecasting performance is compared with observed rainfall values, and evaluated by common statistics of relative absolute error and Nash–Sutcliffe efficiency. The results show that the HWNN model improves the monthly rainfall forecasting accuracy over Australia in comparison to the reference models, and the improvement is more significant for the inland stations in southeast Australia and stations in west Australia.