Backpropagation Learning Research Articles

Grain disintegration and dynamic recrystallization during impact tests of additively manufactured nickel-based alloy 718

The high-temperature dynamic mechanical response of Alloy 718 produced via laser-powder bed fusion (LPBF) was investigated through compressive Split-Hopkinson Pressure Bar (SHPB) tests. Simulating the typical service conditions of Alloy 718, the tests were conducted at temperatures ranging from 298 K to 773 K and at strain rates of 1000 s−1 to 1500 s−1. Phenomenological material constitutive models, such as the modified versions of Johnson-Cook and Hensel-Spittel models, were developed based on the SHPB test results. Analysis of microstructural evolution under impact conditions highlighted that columnar grains with high Schmid factors tend to undergo preferential activation and dislocation pile-up. This process leads to the formation of adiabatic shear bands, grain disintegration, and intense lattice rotation, particularly at higher strain rates. Furthermore, increasing the dynamic deformation temperature facilitates the activation of discontinuous dynamic recrystallization (DRX), with strain accumulation promoting localized grain nucleation along heavily dislocated dendritic boundaries. Recognizing the limitations of phenomenological material constitutive models in accurately representing the underlying microstructural evolution, an artificial neural network (ANN)-based constitutive model employing a three-layer backpropagation learning algorithm was implemented, reducing the Average Absolute Relative Error (AARE) to 0.17%.

Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks

Accumulated detailed knowledge about the neuronal activities in human brains has brought more attention to bio-inspired spiking neural networks (SNNs). In contrast to non-spiking deep neural networks (DNNs), SNNs can encode and transmit spatiotemporal information more efficiently by exploiting biologically realistic and low-power event-driven neuromorphic architectures. However, the supervised learning of SNNs still remains a challenge because the spike-timing-dependent plasticity (STDP) of connected spiking neurons is difficult to implement and interpret in existing backpropagation learning schemes. This paper proposes a fractional-order spike-timing-dependent gradient descent (FO-STDGD) learning model by considering a derived nonlinear activation function that describes the relationship between the quasi-instantaneous firing rate and the temporal membrane potentials of nonleaky integrate-and-fire neurons. The training strategy can be generalized to any fractional orders between 0 and 2 since the FO-STDGD incorporates the fractional gradient descent method into the calculation of spike-timing-dependent loss gradients. The proposed FO-STDGD model is tested on the MNIST and DVS128 Gesture datasets and its accuracy under different network structure and fractional orders is analyzed. It can be found that the classification accuracy increases as the fractional order increases, and specifically, the case of fractional order 1.9 improves by 155 % relative to the case of fractional order 1 (traditional gradient descent). In addition, our scheme demonstrates the state-of-the-art computational efficacy for the same SNN structure and training epochs.

Advancing Autonomous Vehicle Navigation through Hybrid Fuzzy-Neural Network Training Systems

In the evolving realm of autonomous vehicle navigation, the integration of fuzzy logic and neural networks presents a formidable challenge, particularly in the context of real-time, on-the-fly neural network training. This paper addresses the gap in dynamic and adaptable training methods necessary for navigating unpredictable environments with limited computational resources. The primary objective of our study is to empirically validate a hybrid training approach that combines fuzzy logic with back-propagation learning algorithms, aiming to optimize neural network performance under hardware constraints. Our methodology leverages a fuzzy logic trainer to provide initial training sets dynamically, which guide the neural network in adjusting its weights in real time, thus facilitating adaptive learning during navigation tasks. The findings reveal that this integrated approach not only enhances the learning efficiency of neural networks but also significantly improves navigation accuracy in real-time scenarios. These advancements contribute to the field by demonstrating the feasibility of deploying more adaptable and robust autonomous navigation systems, potentially expanding their application in more diverse and challenging environments.

Ternary blended concrete strength evaluation: experimental and artificial intelligence techniques

Purpose The purpose of this study is to forecast the mechanical properties of ternary blended concrete (TBC) modified with oyster shell powder (OSP) and shea nutshell ash (SNA) using deep neural network (DNN) models. Design/methodology/approach DNN models with three hidden layers, each layer containing 5–30 nodes, were used to predict the target variables (compressive strength [CS], flexural strength [FS] and split tensile strength [STS]) for the eight input variables of concrete classes 25 and 30 MPa. The concrete samples were cured for 3–120 days. Levenberg−Marquardt's backpropagation learning technique trained the networks, and the model's precision was confirmed using the experimental data set. Findings The DNN model with a 25-node structure yielded a strong relation for training, validating and testing the input and output variables with the lowest mean squared error (MSE) and the highest correlation coefficient (R) values of 0.0099 and 99.91% for CS and 0.010 and 98.42% for FS compared to other architectures. However, the DNN model with a 20-node architecture yielded a strong correlation for STS, with the lowest MSE and the highest R values of 0.013 and 97.26%. Strong relationships were found between the developed models and raw experimental data sets, with R2 values of 99.58%, 97.85% and 97.58% for CS, FS and STS, respectively. Originality/value To the best of the authors’ knowledge, this novel research establishes the prospects of replacing SNA and OSP with Portland limestone cement (PLC) to produce TBC. In addition, predicting the CS, FS and STS of TBC modified with OSP and SNA using DNN models is original, optimizing the time, cost and quality of concrete.

MATHEMATICAL APPROACH OF THE BACKPROPAGATION METHOD FOR CONSTRUCTING ARTIFICIAL NEURAL NETWORKS

Backpropagation isthe core part of a neural network. This method is used to efficiently train a network using a chain rule that allows differentiation of complex functions. In other words, after each pass through the network, the backpropagation method performs a backward pass to adjust the model parameters, such as weights and biases. This article highlights the importance of using the backpropagation method from the point of view of mathematical formulas for neural networks. The importance of using the backpropaga-tion learning algorithm to calculate the gradient (gradient descent) and the need to use the activation function to minimize the loss function is mathematically described and calculated by formulas, and also proven by calculating the matrix products of vectors for each layer of parameters - weights and biases and applying complex differential equations.

Adaptive QSMO-Based Sensorless Drive for IPM Motor with NN-Based Transient Position Error Compensation

In commercial electrical equipment, the popular sensorless drive scheme for the interior permanent magnet synchronous motor, based on the quasi-sliding mode observer (QSMO) and phase-locked loop (PLL), still faces challenges such as position errors and limited applicability across a wide speed range. To address these problems, this paper analyzes the frequency domain model of the QSMO. A QSMO-based parameter adaptation method is proposed to adjust the boundary layer and widen the speed operating range, considering the QSMO bandwidth. A QSMO-based phase lag compensation method is proposed to mitigate steady-state position errors, considering the QSMO phase lag. Then, the PLL model is analyzed to select the estimated speed difference for transient position error compensation. Specifically, a transient position error compensator based on a feedback time delay neural network (FB-TDNN) is proposed. Based on the back propagation learning algorithm, the specific structure and optimal parameters of the FB-TDNN are determined during the offline training process. The proposed parameter adaptation method and two position error compensation methods were validated through simulations in simulated wide-speed operation scenarios, including sudden speed changes. Overall, the proposed scheme fully mitigates steady-state position errors, substantially mitigates transient position errors, and exhibits good stability across a wide speed range.

A Forward Learning Algorithm for Neural Memory Ordinary Differential Equations.

The deep neural network, based on the backpropagation learning algorithm, has achieved tremendous success. However, the backpropagation algorithm is consistently considered biologically implausible. Many efforts have recently been made to address these biological implausibility issues, nevertheless, these methods are tailored to discrete neural network structures. Continuous neural networks are crucial for investigating novel neural network models with more biologically dynamic characteristics and for interpretability of large language models. The neural memory ordinary differential equation (nmODE) is a recently proposed continuous neural network model that exhibits several intriguing properties. In this study, we present a forward-learning algorithm, called nmForwardLA, for nmODE. This algorithm boasts lower computational dimensions and greater efficiency. Compared with the other learning algorithms, experimental results on MNIST, CIFAR10, and CIFAR100 demonstrate its potency.

Comparative study of integer-order and fractional-order artificial neural networks: Application for mathematical function generation

This paper investigates the impact of fractional derivatives on the activation functions of an artificial neural network (ANN). Based on the results and analysis, a three-layer backpropagation neural network model with fractional and integer derivatives employed in the activation function and fractional gradient descent method backpropagation learning algorithm has been proposed. Specifically, three perceptrons have been proposed based on fractional and integer derivatives applied to activation functions and learning algorithms. They are fractional derivative activation function (FDAF) perceptron, integer derivative activation function (IDAF)-fractional derivative learning algorithm (FDLA) perceptron, and fractional derivative learning algorithm (FDLA) perceptron. The Riemann-Liouville (RL) fractional derivative, Grunwald-Letnikov (GL) derivative, Caputo-Fabrizio (CF) fractional derivative, Caputo (C) fractional derivative, and Atangana-Baleanu (AB) fractional derivative have been employed. The impact of these derivatives’ fractional-order (FO) is investigated in a wide range from 0.1−0.9 on testing mean square error (MSE) and time required to train the perceptron. FO-based and IO-based perceptrons are compared with the help of performance metrics such as testing MSE and the time required to train the models. The training and testing simulation results illustrate that Caputo-Fabrizio derivative-based perceptrons outperform other fractional derivatives. Also, the performance of the FO-based perceptron is better in terms of the least MSE.

The reliability analysis and experiment verification of pressure spherical model for deep sea submersible based on data BP and machine learning technology

Spherical pressure-resistant shells, as a universal structural component of deep-sea submersibles, provide a safe and normal operating environment for personnel and internal equipment. In the paper it presented and optimized the BP neural network model based on a genetic algorithm (GA) accordingly, and the method and accuracy are validated through by a beam model. Simultaneously focusing on steel spherical shells, the study proposed a dataset that captures the influence of the primary dimension of the shell (radius-to-thickness ratio, R/t) on the critical pressure response. The genetic algorithm is employed to optimize the back propagation (BP) neural network model for predicting critical pressure. The structural reliability is adopted as a design criterion to determinate and optimize the geometric parameters and critical pressure of the spherical shell structure. Finally, an ultra-high-strength steel spherical model is designed, constructed and meanwhile collapse pressure tests are accomplished to verify the accuracy of the presented improved BP neural network model based on the computational reliability method. The results reveal that the machine learning optimization design method proposed in this paper can effectively enhance the accuracy of critical pressure predictions and the precision of reliability assessments for deep-sea spherical shells.

Deep Learning Based Over-the-Air Training of Wireless Communication Systems without Feedback.

In trainable wireless communications systems, the use of deep learning for over-the-air training aims to address the discontinuity in backpropagation learning caused by the channel environment. The primary methods supporting this learning procedure either directly approximate the backpropagation gradients using techniques derived from reinforcement learning, or explicitly model the channel environment by training a generative channel model. In both cases, over-the-air training of transmitter and receiver requires a feedback channel to sound the channel environment and obtain measurements of the learning objective. The use of continuous feedback not only demands extra system resources but also makes the training process more susceptible to adversarial attacks. Conversely, opting for a feedback-free approach to train the models over the forward link, exclusively on the receiver side, could pose challenges to reliably end the training process without intermittent testing over the actual channel environment. In this article, we propose a novel method for the over-the-air training of wireless communication systems that does not require a feedback channel to train the transmitter and receiver. Random samples are transmitted through the channel environment to train a mixture density network to approximate the channel distribution on the receiver side of the network. The transmitter and receiver models are trained with the resulting channel model, and the transmitter can be deployed after training. We show that the block error rate measurements obtained with the simulated channel are suitable for monitoring as a stopping criterion during the training process. The resulting method is demonstrated to have equivalent performance to the end-to-end autoencoder training on small message sequences.

Prediction of pressure evolution in non-venting self-pressurized liquid hydrogen tanks using artificial neural network approach

Liquid hydrogen (LH2) is a future energy carrier compared to most conventional fuels. One of the practical bottlenecks in non-venting LH2 tanks is to accurately predict the pressure evolution. Accordingly, many thermal models and correlations were developed; however significant deviations arose due to inherent complex thermal phenomena which have not been addressed yet. A general artificial neural network (ANN) model has been first developed based on 448 datasets amassed from different literature sources to predict the pressure evolution under different heat fluxes, initial filling ratios, as well as tank volumes and shapes. The model has been configured with a typical multi-layer feed-forward structure using a back-propagation learning algorithm and Levenberg-Marquardt training algorithm. For the ANN model, a preliminary optimization process has been conducted for reaching the optimal values for training-to-testing datasets ratio, learning parameters and optimum network structure. Besides, some criteria were set to evaluate the ANN model performance. The ANN model with the configuration of 5-11-11-1 was found to be adopted for such a prediction task. Compared to experimental results and literature models and correlations, the ANN model has been proven to predict the pressure evolution in non-venting self-pressurized LH2 tanks with a mean square error of 1.1779 × 10−5 s2, root mean square error of 0.0034 s, mean absolute error of 6%, mean relative error of 0.2% as well as a satisfactory correlation coefficient of 0.99934 between the network outputs and corresponding targets. This research work provides an effective modeling approach based on artificial neural network to solve one complex problem related with LH2 tanks with fast, accurate and consistent results. Also, the results of the present work can offer some practical guidance for the design of LH2 tanks. Further, the present work gives some insights into new application areas of ANN which will be the emphasis of future research work.

Artificial Neural Network for Automated Keratoconus Detection Using a Combined Placido Disc and Anterior Segment Optical Coherence Tomography Topographer.

To assess the efficacy of an automated program for keratoconus and keratoconus suspect detection based on corneal measurements provided by a combined Placido disc and anterior segment optical coherence tomography (OCT) topographer. In a multicentric cross-sectional study, an artificial neural network (ANN) was created using 6677 eyes from an equal number of patients (classified as 2663 normal eyes, 1616 keratoconus eyes, 210 keratoconus suspect eyes, 1519 myopic postoperative eyes, and 669 abnormal eyes). Each group was randomly divided into a training set (70% of the dataset) and a validation set (the remaining 30%). A multilayer perceptron network with a backpropagation learning algorithm was developed for the study. Indexes used to train the ANN were based on curvature and elevation of both the anterior and posterior corneal surfaces and the new corneal OCT indexes-based on corneal, stromal, and epithelial thicknesses. For keratoconus detection, our ANN showed an accuracy of 98.6%, precision of 96%, recall of 97.9%, and F1-score of 96.9%. For keratoconus suspect detection, our ANN showed an accuracy of 98.5%, precision of 83.6%, recall of 69.7%, and F1-score of 76%. Compared to previous literature, the addition of new OCT-based epithelial and stromal thickness indexes improves ANN detection capacity of keratoconus suspect eyes. For already stablished keratoconus our ANN detection capacity is excellent, but equivalent to previous evidence without incorporating such new OCT-based indexes. OCT-based epithelial and stromal thickness indexes improve ANN detection capacity of keratoconus on its early stages.

Parametric investigation of battery thermal management system with phase change material, metal foam, and fins; utilizing CFD and ANN models

The focus on developing an effective battery thermal management system (BTMS) to maintain optimal temperatures for lithium-ion batteries (LIBs), especially in electric vehicle (EV) applications, has grown significantly. The effective BTMS not only enhances the cooling performance of LIBs but also contributes to increased passenger safety and mileage of EVs. This study investigates BTMS configurations with fins, metal foam, and phase change material (PCM) to minimize temperature of battery during 3C discharging in varying conditions. Additionally, the study explores the impact of different BTMS material combinations and various fins lengths on system performance as a parametric investigation. Moreover, to streamline the analysis process and introduce novelty, artificial intelligence is explored as an alternative to computational fluid dynamics for predicting liquid fraction of PCM and temperature of battery, enhancing the innovative aspect of this study. Numerical simulations, using a non-equilibrium thermal model for metal foam modeling, reveal that the fourth case, integrating all three passive approaches, maintains the lowest temperature and enhances LIB cooling. The optimum BTMS shows a reduction of 3 K compared to BTMS utilizing pure PCM. During discharge process, the temperature difference in the battery decreases by approximately 75 % and 66 % in the fourth case compared to the first case (with pure PCM) under normal and harsh environmental conditions, respectively. Applying copper metal foam and copper fins yields the best results in reducing battery temperature. Increasing the length of fins and adding more fins effectively lower the battery temperature. Finally, an artificial neural network model is developed using the backpropagation learning technique coupled with the gradient descent optimization algorithm. The model exhibits excellent predictive capabilities, achieving high R-squared values of 0.98 for PCM liquid fraction and 0.99 for battery temperature.

Machine learning backpropagation network analysis of permeability, Forchheimer coefficient, and effective thermal conductivity of macroporous foam–fluid systems

Macroporous materials exhibit outstanding properties in heat and mass transfer due to their high pore volume, high surface area, and high Young's modulus. Consequently, understanding their thermofluidic properties is crucial in the design, synthesis, and optimal application of these materials. Therefore, this study, premieres, the use of a machine learning (ML) backpropagation network to develop and train a series of datasets for permeability, Forchheimer coefficient, and effective thermal conductivity of variable macroporous foam–fluid systems with respect to degrees of interstices, fluid and solid properties. To account for permeability values for flowing fluids in the Darcy regime, numerical simulations of slow–moving fluids were implemented over the materials' interstices. In comparison to similarly substantiated values of permeability in the Forchheimer regime, these values were a bit lower. The ML-based backpropagation algorithm was used to analyze data, which produced predictions (output signals) that are more than 90 % in correlation to CFD datasets. This provided insight into the effect of porosity and reduced mean pore openings on macroporous structures' thermofluidic behaviour. Material porosity was observed to play a dominant role in estimating Forchheimer coefficients and effective thermal conductivities for these foam-fluid systems. However, reduced mean pore openings were observed to be more critical for estimating permeability. The contributory effects of reduced mean pore openings on the effective thermal conductivity for these macroporous foam–fluid systems were determined to vary between 5.8 and 13.2 percent. Furthermore, the effective thermal conductivity of macroporous foam–fluid systems was also evaluated in relation to changes in the interstitial fluid and solid matrix thermal conductivity.

Design of backpropagation learning algorithm for MHD mixed convective Prandtl nanofluid flow with activation energy

Design of backpropagation learning algorithm for MHD mixed convective Prandtl nanofluid flow with activation energy

An applied deep reinforcement learning approach to control active networked microgrids in smart cities with multi-level participation of battery energy storage system and electric vehicles

This study proposed an intelligent energy management strategy for islanded networked microgrids (NMGs) in smart cities considering the renewable energy sources uncertainties and power fluctuations. Energy management of active power and frequency control approach is based on the intelligent probabilistic wavelet fuzzy neural network-deep reinforcement learning algorithm (IPWFNN-DRLA). The control strategy is formulated with deep reinforcement learning approach based on the Markov decision process and solved by the soft actor-critic algorithm. The NMG local controller (NMGLC) provides information such as the frequency, active power, power generation data, and status of the electric vehicle's battery energy storage system to the NMG central controller (NMGCC). Then the NMGCC calculates active power and frequency support based on the IPWFNN-DRLA approach and sends the results to the NMGLC. The proposed model is developed in a continuous problem-solving space with two structures of offline training and decentralized distributed operation. For this purpose, each NMG has a control agent (NMGCA) based on the IPWFNN algorithm, and the NMGCA learning model is formulated based the online back-propagation learning algorithm. The proposed approach demonstrates a computation accuracy exceeding 98%, along with a 7.82% reduction in computational burden and a 61.1% reduction in computation time compared to alternative methods.

Machine Learning Backpropagation Prediction and Analysis of the Thermal Degradation of Poly (Vinyl Alcohol).

Thermogravimetric analysis (TGA) is crucial for describing polymer materials' thermal behavior as a result of temperature changes. While available TGA data substantiated in the literature significantly focus attention on TGA performed at higher heating rates, this study focuses on the machine learning backpropagation analysis of the thermal degradation of poly (vinyl alcohol), or PVA, at low heating rates, typically 2, 5 and 10 K/min, at temperatures between 25 and 600 °C. Initial TGA analysis showed that a consistent increase in heating rate resulted in an increase in degradation temperature as the resulting thermograms shifted toward a temperature maxima. At degradation temperatures between 205 and 405 °C, significant depths in the characterization of weight losses were reached, which may be attributed to the decomposition and loss of material content. Artificial neural network backpropagation of machine learning algorithms were used for developing mathematical descriptions of the percentage weight loss (output) by these PVA materials as a function of the heating rate (input 1) and degradation temperature (input 2) used in TGA analysis. For all low heating rates, modelling predictions were observably correlated with experiments with a 99.2% correlation coefficient and were used to interpolate TGA data at 3.5 and 7.5 K/min, indicating trends strongly supported by experimental TGA data as well as literature research. Thus, this approach could provide a useful tool for predicting the thermograms of PVA materials at low heating rates and contribute to the development of more advanced PVA/polymer materials for home and industrial applications.

Deep Image Feature Learning With Fuzzy Rules

Feature extraction methods are key to many image processing tasks. At present, the most popular method is to use a deep neural network to automatically extract features through end-to-end training instead of the traditional hand-crafted feature extraction. However, the training of deep neural network relies heavily on data quality and quantity, and the network is a black-box model that has poor interpretability. Human intelligence can be leveraged here to improve the deep neural network model, where the human decision process can be integrated in feature learning and object classification to enhance robustness and interpretability. In this paper, the method Deep Image Feature Learning with Fuzzy Rules (DIFL-FR) is proposed, where human decision process is embedded in feature extraction by combining fuzzy logic rule-based modeling with deep-stacked learning strategy. The proposed method has the following distinctive characteristics. First, since the method is based on fuzzy sets and fuzzy inference, it can extract more robust features from noisy data scenes. Second, the method progressively learns the image features through a layer-by-layer approach based on fuzzy rules, so that the feature learning process can be better explained by the rules generated. Third, the learning process of the method has a high efficiency since it is only based on forward propagation without back propagation and iterative learning. Finally, while the method is based on unsupervised learning, it can be easily extended to supervised and semi-supervised learning cases. The results of extensive experiments conducted on image datasets of different scales clearly show the effectiveness of the proposed method.

Simultaneous spectrophotometric determination of Co (II) and Co (III) in acidic medium with partial least squares regression and artificial neural networks

This study aims at the application of two chemometric techniques to visible spectra of acetic acid solutions of Co (II) and Co (III) for simultaneous determination thereof. Spectral data of 145 samples in the range of 400–700 nm were used to build the models. Partial least squares regression models were developed for which latent variables were determined using internal cross-validation with a leave-one-out strategy and 3 and 2 latent variables were selected for Co(II) and Co(III) based on root mean square error of cross-validation. For these models, root mean square errors of prediction were 1.16 and 0.536 mM and coefficients of determination were 0.975 and 0.892 for Co (II) and Co (III). As an alternate method, artificial neural networks consisting of three layers, with 10 neurons in hidden layer, were trained to model spectra and concentrations of cobalt species. Levenberg-Marquardt algorithm with feed-forward back-propagation learning resulted root mean square errors of prediction of 0.316 and 0.346 mM for Co (II) and Co (III) respectively and coefficients of determination were 0.996 and 0.988.

Empirical study of college students' extracurricular reading preference by functional data analysis of the library book borrowing behavior.

Library data contains many students' reading records that reflect their general knowledge acquisition. The purpose of this study is to deeply mine the library book-borrowing data, with concerns on different book catalogues and properties to predict the students' extracurricular interests. An intelligent computing framework is proposed by the fusion of a neural network architecture and a partial differential equations (PDE) function module. In model designs, the architecture is constructed as an adaptive learning backpropagation neural network (BPNN), with automatic tuning of its hyperparameters. The PDE module is embedded into the network structure to enhance the loss functions of each neural perceptron. For model evaluation, a novel comprehensive index is designed using the calculus of information entropy. Empirical experiments are conducted on a diverse and multimodal time-series dataset of library book borrowing records to demonstrate the effectiveness of the proposed methodology. Results validate that the proposed framework is capable of revealing the students' extracurricular reading interests by processing related book borrowing records, and expected to be applied to "big data" analysis for a wide range of various libraries.