Neurons In Hidden Layer Research Articles

Correlation-based pruning algorithm with weight compensation for feedforward neural networks

Abstract Optimizing neural network architectures through effective pruning techniques has become essential to balancing model complexity and accuracy. This study introduces a novel correlation-based approach to systematically reduce network size by identifying and removing redundant neurons based on their activation correlations. By selectively pruning neurons while compensating for their contributions, the method maintains model fidelity across diverse datasets. Results demonstrate substantial architecture reductions with minimal performance impact: For the MNIST dataset, the number of neurons in hidden layers was reduced from 128-128 to 118-93, while maintaining a high accuracy of 97.59%. Comparative analysis indicates that this pruning approach achieves competitive or superior results compared to state-of-the-art methods while reducing computational complexity and memory requirements by up to 25%. The findings highlight the potential of correlation-driven pruning strategies to optimize neural networks, making them more efficient and adaptable to resource-constrained environments.

Closed-form interpretation of neural network classifiers with symbolic gradients

Abstract I introduce a unified framework for finding a closed-form interpretation of any single neuron in an artificial neural network. Using this framework I demonstrate how to interpret neural network classifiers to reveal closed-form expressions of the concepts encoded in their decision boundaries. In contrast to neural network-based regression, for classification, it is in general impossible to express the neural network in the form of a symbolic equation even if the neural network itself bases its classification on a quantity that can be written as a closed-form equation. The interpretation framework is based on embedding trained neural networks into an equivalence class of functions that encode the same concept. I interpret these neural networks by finding an intersection between the equivalence class and human-readable equations defined by a symbolic search space. The approach is not limited to classifiers or full neural networks and can be applied to arbitrary neurons in hidden layers or latent spaces.

Prediction acrylamide contents in fried dough twist based on the application of artificial neural network

Acrylamide forms through the reaction between reducing sugars and asparagine in the thermal processing of food. Detection measures like LC-MS, HPLC are time-consuming and costly, which inspired us to use back propagation-artificial neural networks (BP-ANN) based on a genetic algorithm to establish an acrylamide prediction model in fried dough twist. The effects of frying time and temperature on acrylamide contents, as well as the color difference and acid value at different time and temperature were determined. Acrylamide content was found significantly correlated with temperature (P < 0.01) and was correlated with acid value and color difference (P < 0.05). Thus, temperature, acid value, and the color difference were set as input layers, and acrylamide content was set as an output layer to establish a BP-ANN network prediction model. The weight and threshold values in the BP-ANN network prediction model were optimized with a multi-population genetic algorithm and the test data were set to obtain an optimized BP neural network predicting model. The results showed that the Levenberg-Marquardt back-propagation training algorithm of the BP-ANN model with 5 hidden layer neurons and 0.005 learning rate was the best predictive performance, which the correlation coefficients (R) of test and validation were 0.9640 and 0.8999, suggesting a good fitting and strong approximation ability. The BP-ANN model is expected to accurately predict the content of acrylamide in fried dough twist.

Using Machine Learning for Climate Modelling: Application of Neural Networks to a Slow-Fast Chaotic Dynamical System as a Case Study

This paper explores the capabilities of two types of recurrent neural networks, unidirectional and bidirectional long short-term memory networks, to build a surrogate model for a coupled fast–slow dynamic system and predicting its nonlinear chaotic behaviour. The dynamical system in question, comprising two versions of the classical Lorenz model with a small time-scale separation factor, is treated as an atmosphere–ocean research simulator. In numerical experiments, the number of hidden layers and the number of nodes in each hidden layer varied from 1 to 5 and from 16 to 256, respectively. The basic configuration of the surrogate model, determined experimentally, has three hidden layers, each comprising between 16 and 128 nodes. The findings revealed the advantages of bidirectional neural networks over unidirectional ones in terms of forecasting accuracy. As the forecast horizon increases, the accuracy of forecasts deteriorates, which was quite expected, primarily due to the chaotic behaviour of the fast subsystem. All other things being equal, increasing the number of neurons in hidden layers facilitates the improvement of forecast accuracy. The obtained results indicate that the quality of short-term forecasts with a lead time of up to 0.75 model time units (MTU) improves most significantly. The predictability limit of the fast subsystem (“atmosphere”) is somewhat greater than the Lyapunov time.

Modelling and optimizing secondary metabolites production in Spirodela polyrhiza using machine learning

Spirodela. polyrhiza as one of the potential feedstocks for industrial natural dye and pharmaceutical products, efficient process to produce from it were scarce. Here, Artificial Neural Network (ANN) and Genetic Algorithm (GA) were employed for modelling and optimization respectively. Current study focus on establishing useable models and anticipating the optimal concentration of nitrate (NO3−), phosphate (PO43−) and ammonium (NH4+) to give highest relative growth rate (RGR) and produce maximum amount of anthocyanins, chlorophylls, phenolic compounds and antioxidants. In short, 12 and 11 neurons in hidden layer were found to be most suitable neuron numbers used for the modelling of growth and secondary metabolites respectively. All the models exhibited a high degree of robustness and performance with a high r2 value (higher than 0.75) and a low mean squared error (lower than 0.05). GA also anticipated the best concentration of macronutrients to achieve the maximum growth and metabolites production with significant low errors (below 7 %). The employment of machine learning was useful for such applications.

Research on Pattern Classification Based on Double Pseudo-Inverse Extreme Learning Machine

This research aims to address the limitations inherent in the traditional Extreme Learning Machine (ELM) algorithm, particularly the stochastic determination of input-layer weights and hidden-layer biases, which frequently leads to an excessive number of hidden-layer neurons and inconsistent performance. To augment the neural network’s efficacy in pattern classification, Principal Component Analysis (PCA) is employed to reduce the dimensionality of the input matrix and alleviate multicollinearity issues during the computation of the input weight matrix. This paper introduces an enhanced ELM methodology, designated the PCA-DP-ELM algorithm, which integrates PCA with Double Pseudo-Inverse Weight Determination (DP). The PCA-DP-ELM algorithm proposed in this study consistently achieves superior average classification accuracy across various datasets, irrespective of whether assessed through longitudinal or cross-sectional experiments. The results from both experimental paradigms indicate that the optimized algorithm not only enhances accuracy but also improves stability. These findings substantiate that the proposed methodology exerts a positive influence on pattern classification.

An aeration requirements calculating method based on BOD5 soft measurement model using deep learning and improved coati optimization algorithm

Conventionally, the supply-side is over-aerated to meet the oxygen demand of biological wastewater treatment in wastewater treatment plants (WWTPs). Such systems do not consider the various influent loadings and actual oxygen consumption. It cannot respond effectively to fluctuating influent, resulting in unnecessary aeration and energy wastage. In this study, we developed a novel approach to calculate aeration requirements in real time. Firstly, the oxygen demand formula for biological nitrogen and phosphorus removal is modified by considering realistic factors (temperature, oxygen partial pressure and so on) to construct the aeration demand formula. Then, to address the problem that BOD5 in aeration formula is difficult to measure in real time, a novel hybrid model integrating Fuch mapping, reverse learning strategy, Coati Optimization Algorithm (COA), and Backpropagation Artificial Neural Network (BP-ANN) is proposed in this study. Finally, aeration requirements are calculated based on real-time water quality data. Experimental results show that the ICOA-BP model with 9 neurons in hidden layer predicted the optimum BOD5 concentration by achieving a 25.146 mean absolute error and a 0.735 coefficient of determination (all-R2) in all datasets. The optimization led to a 15.164 % aeration savings and 38.942 % energy savings. The proposed method of aeration calculation is automated and intelligent, responding to fluctuating changes in influent quality and reducing unnecessary aeration and energy consumption while meeting the requirements of effluent quality.

Improving milling tool wear prediction through a hybrid NCA-SMA-GRU deep learning model

Milling tool wear, a ubiquitous challenge in industrial automation and manufacturing, leads to diminished equipment utilization, escalating costs, and a decline in product quality. The prediction of tool wear is a complex and challenging task, as it involves numerous variables. This paper introduces a pioneering hybrid approach, the NCA-SMA-GRU model. It involves the hybridization of three major components and is designed to enhance the precision and expedite the process of tool wear prediction. The NCA component is adept at filtering and retaining the most relevant features associated with milling tool wear from the raw signals, and it also improves the model’s interpretability. Subsequently, SMA optimizes the GRU network’s hyperparameters, including the initial learning rate, hidden layer neurons, network training iterations, and the L2 regularization factor, to identify an optimal combination that bolsters predictive performance. The modeling steps and the development of fitness function are explained in detail. The model’s efficacy is rigorously evaluated using data from the 2010 High Speed CNC Machine Tool Health Prediction Contest (PHM 2010), which encompasses wear data from both single and multiple cutters. The performance is compared using metrics such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE), R-Squared (R2), and computational time. To assess the ability and characteristics of the proposed approach, other popular hybrid models are constructed for comparative analysis. The results demonstrate that the proposed model addresses the limitations of traditional prediction methods and provides insights into the development of deep learning and optimization algorithms for tool wear prediction.

Design of an artificial neural network and proportional‐integral‐derivative controller using particle swarm optimization for Boeing 747‐400 aircraft pitch control

AbstractThis paper presents the design of an artificial neural network (ANN) and proportional integral derivative (PID) controller using particle swarm optimization (PSO) for Boeing 747‐400 aircraft pitch control (APC). The combinations of disturbance, open loop unstable and nonlinear dynamics are major problems in a Boeing 747‐400 commercial aircraft. This paper investigates the control mechanism of pitch angle control of Boeing 747‐400 with small disturbance theory linearization methods and ANN based non‐linear controllers. A PID controller is tuned by PSO, whereas the PID is tuned by graphical user interface (GUI) when compared with an ANN controller. The controller for this system was designed using an ANN controller and PID tuned using a recent optimization technique such as the PSO method with integral square error (ISE) as an objective function. A comparative study of the time domain performances of the pitch control of the Boeing 747‐400 commercial aircraft was presented. The ANN controller outperformed the PID‐PSO and PID‐GUI controllers in terms of system performance, including rising time (tr), settling time (ts), percentage overshoot (percent OS), and steady state error, across various elevator deflection angles. Basically, the percentage overshoot and steady state error were 0% and 0 respectively, indicating that the ANN controller achieved an improvement of 100%. Various parameters were compared with the PID‐GUI, PID‐PSO, and ANN controllers for pitch control of the Boeing 747‐400 air craft. The ANN controller architecture comprises of two input neurons, two hidden layer neurons, and one output layer neuron. The simulation was performed using Matlab/Simulink. The results show that the PID‐PSO controller was improved by the ANN controller and the performance specifications of the aircraft obtained by the ANN controller were satisfactory.

The performance analysis of deep learning algorithms for modelling and forecasting the particulate matter (PM10) in the eastern part of Turkey

The main purpose of this study is to select the most reliable nonlinear computational model to predict the particulate matter (PM10) concentrations. Time series data of three years PM10 concentrations were used as input variable. For the prediction, three different types of dynamic nonlinear autoregressive models were built and compared. These models are the Levenberg-Marquardt algorithm, the Bayesian Regulization algorithm, and the Scaled Conjugate Gradient algorithm. For each of these algorythms, various settings were adopted with the subsequent statistical analysis. To analyse the model performance, we used mean prediction error, root mean square error, and correlation coefficient. The lowest root mean square errors were found for the Levenberg-Marquardt algorithm with 15 neurons, for the Bayesian Regularization and for the Scaled Conjugate Gradient algorithm with 20 neurons in hidden layer. In our study, we focused on the long-term forecast of stationary dynamic time series data and on the large amount of data, which is presented as a scientific novelty. Additionally, we determined the main model parameters that most improve quality in terms of training and network capacity. Therefore, the derived forecasting model can be used as a priori for air quality management and regulations aimed on the reducing of pollutant level.

Data-driven health state estimation and remaining useful life prediction of fuel cells

Proton exchange membrane fuel cells (PEMFCs) can revolutionise transportation energy and promote environmentally friendly development. The purpose of this study is to predict the state of health (SOH) of PEMFCs and provide guidance for fuel cell maintenance. Under changing power demand situations, a practical method based on the Fréchet distance is proposed to predict the SOH, along with an empirical model to differentiate between the running-in and degradation periods. The proposed method does not require complex and expensive testing instruments and has a relative error of approximately 4.3 %. A voltage drop prediction model is established for steady power demand situations using the particle swarm optimisation-extreme learning machine (PSO-ELM) algorithm. Different activation functions and hidden layer neurons are investigated to enhance prediction accuracy. This study shows that the model effectively tracks the decreasing trend in the transmission voltage of the PEMFC stack. Additionally, a comprehensive analysis framework is developed to address the issue of the possibility of missing system parameters in practical applications. The influence of the system parameters on voltage drop prediction is thoroughly analysed, and the necessary parameters for accurate prediction are defined, providing theoretical guidance for practical monitoring and data collection.

Enhancing urban blue-green landscape quality assessment through hybrid genetic algorithm-back propagation (GA-BP) neural network approach: a case study in Fucheng, China

This study employs an artificial neural network optimization algorithm, enhanced with a Genetic Algorithm-Back Propagation (GA-BP) network, to assess the service quality of urban water bodies and green spaces, aiming to promote healthy urban environments. From an initial set of 95 variables, 29 key variables were selected, including 17 input variables, such as water and green space area, population size, and urbanization rate, six hidden layer neurons, such as patch number, patch density, and average patch size, and one output variable for the comprehensive value of blue-green landscape quality. The results indicate that the GA-BP network achieves an average relative error of 0.94772%, which is superior to the 1.5988% of the traditional BP network. Moreover, it boasts a prediction accuracy of 90% for the comprehensive value of landscape quality from 2015 to 2022, significantly outperforming the BP network’s approximate 70% accuracy. This method enhances the accuracy of landscape quality assessment but also aids in identifying crucial factors influencing quality. It provides scientific and objective guidance for future urban landscape structure and layout, contributing to high-quality urban development and the creation of exemplary living areas.

Recurrent neural network for pitch control of variable-speed wind turbine.

Wind is one of the most widely used renewable energy sources due to its cost-effectiveness, power requirements, operation, and performance. There are many challenges in wind turbines, such as wind fluctuation, pitch control, and generator speed control. When the wind speed exceeds its rated value, the pitch angle controller limits the generator output power to its rated value. In this research work, several soft computing techniques have been implemented for pitch control of variable-speed wind turbine. The data is collected for the National Renewable Energy Laboratory offshore 5 MW baseline wind turbine. Wind speed, tip speed ratio, and power coefficient are taken as inputs, and pitch angle as output. Machine learning and artificial intelligence-based techniques such as recurrent neural networks (RNNs), adaptive neuro-fuzzy inference system (ANFIS), multilayer perceptron feed-forward neural network (MLPFFNN), and fuzzy logic controller (FLC) are implemented on MATLAB, and their results are evaluated in terms of mean square error (MSE) and root mean square error (RMSE). The controllers have been implemented in MATLAB/Simulink to schedule the wind turbine blade pitch angle and keep the output power stable at the rated value. The experimental results show that RNN provided the best results for 15 neurons in hidden layers and 1000 epochs with MSE of 3.28e-11 and RMSE of 5.54e-06, followed by MLPFFNN with MSE of 2.17e-10 and RMSE of 1.56e-05, ANFIS with MSE of 8.5e-05 and RMSE of 9.22e-03, and FLC with MSE of 6.25e-04 and RMSE of 0.025. The proposed scheme is more reliable and robust and can be easily implemented on a physical setup by using interfacing cards such as dSPACE, NI cards, and data acquisition cards.

Forecasting Maximum Water Level Data for Post Sangkuliman using An Artificial Neural Network Backpropagation Algorithm

Neural Network (NN) is an information processing system that has characteristics similar to biological neural networks. One of the algorithms in NN is Backpropagation Neural Network (BPNN). BPNN is an excellent method for dealing with complex pattern recognition problems. In this research, maximum water level forecasting was carried out at Sangkuliman Post using a Backpropagation Neural Network. This research aims to obtain modeling for forecasting maximum water level, as well as forecasting results using the best model. The research results show that the best model is five neurons in hidden layer 1 and 3 neurons in hidden layer 2 with the backpropagation algorithm, the activation function used is binary sigmoid, the learning rate is 0.001, and the maximum iteration is 10,000,000 with the smallest RMSE result being 1.816. The forecast results for the following 12 periods are 1.672, 1.779, 1.523, 1.271, 1.752, 1.692, 1.335, 1.479, 1.750, 1.779, 1.340, 1.269, and 1.754. Forecasting results can be used by various parties in decision-making and planning in multiple fields, as an example to see the patterns of biological and vegetable life around Sangkuliman Post. Based of forecasting results, certain months show an increase in maximum water levels.

Unsupervised damage assessment under varying ambient temperature based on an adjusted artificial neural network and new multivariate covariance-based distances

Temperature variability is one of the critical environmental conditions that causes confusing changes in structural properties and dynamic responses of bridges similar to damage. In this case, false alarms and mis-detection are among the major errors in health monitoring of such civil structures. High damage detectability is another significant challenge in bridge health monitoring. To deal with these issues, this article proposes an unsupervised damage assessment technique comprising two steps of data normalization and novelty detection. For the first step, an adjusted artificial neural network is considered to remove the effects of temperature variability from dynamic features (modal frequencies). This process is carried out by an auto-associative neural network by adjusting its hidden layer neurons through a new hyperparameter selection algorithm. Using normalized features obtained from the first step, this article proposes three multivariate covariance-based distances called linear dissimilarity analysis, multivariate Kullback–Leibler divergence, and multivariate Bregman distance to compute damage indices or novelty scores for damage assessment. The fundamental principles of these distances lie in three aspects: dividing the normalized features into segments, estimating the covariances of segmented feature sets, and incorporating the estimated covariances into the proposed distance measures. The major contributions of this article include proposing three non-parametric distance measures and developing an unsupervised data normalization framework via a new hyperparameter tuning algorithm for adjusting an artificial neural network. A concrete box-girder bridge is considered to verify the proposed approach, along with several comparative studies. Results show that the method presented here can mitigate severe temperature variability and increase damage detectability with superiority over some traditional and state-of-the-art damage assessment techniques.

Simultaneous spectrophotometric determination of Co (II) and Co (III) in acidic medium with partial least squares regression and artificial neural networks

This study aims at the application of two chemometric techniques to visible spectra of acetic acid solutions of Co (II) and Co (III) for simultaneous determination thereof. Spectral data of 145 samples in the range of 400–700 nm were used to build the models. Partial least squares regression models were developed for which latent variables were determined using internal cross-validation with a leave-one-out strategy and 3 and 2 latent variables were selected for Co(II) and Co(III) based on root mean square error of cross-validation. For these models, root mean square errors of prediction were 1.16 and 0.536 mM and coefficients of determination were 0.975 and 0.892 for Co (II) and Co (III). As an alternate method, artificial neural networks consisting of three layers, with 10 neurons in hidden layer, were trained to model spectra and concentrations of cobalt species. Levenberg-Marquardt algorithm with feed-forward back-propagation learning resulted root mean square errors of prediction of 0.316 and 0.346 mM for Co (II) and Co (III) respectively and coefficients of determination were 0.996 and 0.988.

Developing an intelligent prediction system for successful aging based on artificial neural networks.

Due to the growing number of disabilities in elderly, Attention to this period of life is essential to be considered. Few studies focused on the physical, mental, disabilities, and disorders affecting the quality of life in elderly people. SA1 is related to various factors influencing the elderly's life. So, the objective of the current study is to build an intelligent system for SA prediction through ANN2 algorithms to investigate better all factors affecting the elderly life and promote them. This study was performed on 1156 SA and non-SA cases. We applied statistical feature reduction method to obtain the best factors predicting the SA. Two models of ANNs with 5, 10, 15, and 20 neurons in hidden layers were used for model construction. Finally, the best ANN configuration was obtained for predicting the SA using sensitivity, specificity, accuracy, and cross-entropy loss function. The study showed that 25 factors correlated with SA at the statistical level of P < 0.05. Assessing all ANN structures resulted in FF-BP3 algorithm having the configuration of 25-15-1 with accuracy-train of 0.92, accuracy-test of 0.86, and accuracy-validation of 0.87 gaining the best performance over other ANN algorithms. Developing the CDSS for predicting SA has crucial role to effectively inform geriatrics and health care policymakers decision making.

Mass flow rate prediction of a direct-expansion ice thermal storage system using R134a based on dimensionless correlation and artificial neural network

Accurately predicting the refrigerant mass flow rate through the electronic expansion valve (EEV) is crucial for improving the system's performance and achieving intelligent control. However, the refrigerant mass flow rate model applicable to the direct-expansion ice thermal storage (DX-ITS) system for a wide range of flow rates is rare in the open literature. In this study, Buckingham-π theorem and artificial neural network (ANN) are adopted to predict the refrigerant mass flow rate through the EEV of the DX-ITS system using R134a. The dimensionless π-groups and optimal number of neurons in hidden layers of ANN model are obtained. The obtained ANN model shows good accuracy, and over 95.7 % of the predicted data are within the 15 % error band. Results indicate that the EEV outlet pressure has a significant impact on the refrigerant mass flow rate compared with the inlet pressure, the average increase in refrigerant mass flow rate is 43.76 kg/h with an outlet pressure increase of 0.05 MPa. Moreover, the refrigerant mass flow rate gently rises with the increase of superheat temperature under fixed EEV inlet and outlet pressure. On average, every 2 °C increase in superheat temperature leads to an approximately 3.98 kg/h increase in refrigerant mass flow rate.

Short-term load forecasting using an Artificial Neural Network for Battery Energy Storage System

Load levelling that use recent spotlighted the Battery Energy Storage System (BESS) saves spare electricity or low cost electricity. The saved electricity can be used for peak power demand time. In this paper, load is predicted for purpose of operating the BESS effectively. By analysing pattern of industrial load, it is classified into workdays, Saturday and holidays. Then apply ANN on workdays load pattern from classified load patterns to predict load. To increase accuracy, number of neurons in hidden layer and learning data period are changed for estimation of load. After that, error rate was calculated for comparison and analysis.

Numerical investigations of the fractional order derivative-based accelerating universe in the modified gravity

In this work, a Liouville–Caputo fractional order (FO) derivative for the mathematical system based on the accelerating universe in the modified gravity (AUMG), i.e. FO-AUMG is proposed to get more accurate solutions. The nonlinear dynamics of the FO-AUMG is classified into five dynamics. The performances of the designed nonlinear FO-AUMG are numerically stimulated with the stochastic procedures of Levenberg–Marquardt backpropagated (LMB) scheme-based neural networks. The statics for FO-AUMS is used for the nonlinear FO-AUMG as 72%, 16% and 12% for training, authorization, and testing. Twenty neurons in hidden layers have been used to approximate the solution of the nonlinear FO-AUMS. The comparison of three different cases of the nonlinear FO-AUMS is performed with dataset generated by Adams method. To validate the uniformity, legitimacy, precision, and competence of LMB-based adaptive neural networks, the outcomes of the state transitions parameters, regression, correlation, error-histogram plots have been exploited.