Neural Network Training Algorithm Research Articles

Improving the predictive accuracy of artificial neural network (ANN) approach in a mild steel turning operation

Improving and increasing metal cutting process efficiency requires an in-depth knowledge of the cutting process, as well as the development/application of an efficient modeling technique, capable of forming the core tools for planning, scheduling, and integration of the machining processes. This study evaluates the application of Levenberg Marquardt (LM) and Scaled Conjugate Gradient (SCG) training algorithms of artificial neural networks in the modeling and prediction of metal removal rate (MRR) and average surface roughness (Ra) in the turning of cylindrical mild steel. The neural network models were trained using a set of experimental data consisting of the input parameters: cutting speed, depth of cut, and feed rate. Based on the graphical and statistical results, the LM offered the best accuracy in terms of the model fitting, RMSE and R2 obtained. For the LM algorithms, the least RMSE values of 1.7809 × 10−1, 2.6805 × 10−1, and 1.5815 × 10−1 were obtained for the training, validation, and test data respectively, and for the SCG algorithm, least RMSE values of 1.7809 × 10−1, 2.6805 × 10−1, and 1.5815 × 10−1 were recorded. Comparing both algorithms, the Levenberg Marquardt algorithm recorded the least average in terms of the number of the epoch, and RMSE value and at the same time the best performance in terms of good fitting.

ПРОГНОЗИРОВАНИЕ И УПРАВЛЕНИЕ ПРОПУСКНОЙ СПОСОБНОСТЬЮ ПОЛОСЫ НА РЕГУЛИРУЕМОМ ПЕРЕКРЕСТКЕ

Accurate forecasting of traffic flow parameters in real-time is the basis for making decisions about dynamic lane management, which plays an important role in reducing congestion. The existing methods are not able to remember long-term dependencies to obtain an accurate result of predicting the sequence. In our study, we focused on the development of an algorithm for adaptive adjustment of signal control cycles, which ensures the passage of all group vehicles based on the use of R-CNN and YOLOv4 neural networks. In this study, we used surveillance cameras with a large viewing angle to predict the traffic flow. In the process of learning, long-run short-term memory and a recurrent neural network were adapted. The training algorithms of neural networks take into account the dynamic dimensions of vehicles; discrete parameters of the queue before the intersection, and the duration of the cycle. The result of the study was the development of an algorithm for adaptive adjustment of the duration of the resolving cycle of a traffic light object, taking into account the parameters of the traffic flow in the tasks of eliminating or minimizing the possibility of a traffic jam

The approach to increasing the efficiency of convolution algorithms in modern high-availability systems

Most modern high-availability information systems are either based on systems that are classified as artificial intelligence, or to a large extent include them as components. The solution of many problems of artificial intelligence is based on the use of algorithms that implement the convolution operation (for example, algorithms for training neural networks). The article proposes an approach that, on the one hand, provides a strict formalization of this operation based on the algebra of multidimensional matrices, and on the other hand, due to this, it provides such practical advantages as simplifying and reducing the cost of developing such information systems, as well as reducing the execution time of queries to them. due to the simplicity of the development of parallel algorithms and programs and the efficient use of parallel computing systems. Convolution is an indispensable operation for solving many scientific and technical problems, such as machine learning, data analysis, signal processing, image processing filters. An important role is played by multidimensional convolutions, which are widely used in various subject areas. At the same time, due to the complexity of the algorithms that implement them, in practice even three-dimensional convolutions are used much less frequently than one- and two-dimensional ones. Replacing a multidimensional convolution operation with a sequence of convolution operations of lower dimensions significantly increases its computational complexity. The main reason for this lies in the absence of a unified strict definition of the operation and overload in mathematics of the term «convolution». Therefore, the article discusses a multidimensional-matrix computation model, which allows one to effectively formalize problems whose solution uses multidimensional convolution operations, and to implement an effective solution to these problems due to the natural parallelism inherent in the operations of the algebra of multidimensional matrices.

Resevoir property modelling with seismic attributes and artificial neural network

This paper focuses on the analysis of seismics attributes and the application of training algorithms for artificial neural networks to build lithofacies model based on which porosity distribution across a reservoir is modeled. Firstly, seismic attribute and facies log, porosity log are modeled using the standard procedure of Petrel software (Schlumberger). After that, the resulting models are extracted and used as an input data for SOM-supervised algorithm. The result of this step is a map showing the relationship beetwen seismic attributes and facies. In the next step, the map is used to build 3D facies model. With the same procedure, the 3D porosity model is build by Fitnet algorithm. In this work, ANN training, facies modeling and porosity modeling were implemented with MATLAB and the resulting models were compared to the ones that resulted from Petrel software. The good agreement in the porosity distribution patterns between the two models shows that the computational background used in this research is similar to that of Petrel software. The paper contributes to new insights into the fundamentals of computational algorithms used in Petrel which has not been thoroughly studied in Vietnam, and thus helps improve the software usage in reservoir properties modeling.

CNNFOIL: convolutional encoder decoder modeling for pressure fields around airfoils

In this study, we propose an encoder–decoder convolutional neural network-based approach for estimating the pressure field around an airfoil. The developed tool is one of the early steps of a machine-learning-based aerodynamic performance prediction tool. Network training and evaluation are performed from a set of computational fluid dynamics (CFD)-based solutions of the 2-D flow field around a group of known airfoils involving symmetrical, cambered, thick and thin airfoils. Reynolds averaged Navier Stokes-based CFD simulations are performed at a selected single Mach number and for an angle of attack condition. The calculated pressure field, which is the main parameter for lift and drag calculations, is fed to the neural network training algorithm. Pressure data are calculated using CFD methods on high-quality structured computational grids. For the better shape learning, a distance map is generated from airfoil shape and provided to the algorithm at data locations of the pressure points relative to the airfoil shape. Experiments are conducted with unseen airfoil shapes to evaluate the predictive capability of our model. Performance analysis for airfoils with different thicknesses and cambers is conducted. We also investigated the effect of the shock on the performance of our model. Overall, our model achieves 88 $$\%$$ accuracy for unseen airfoil shapes and shows promise to capture the overall flow pattern accurately. Also, significant speed-up is achieved compared to time-consuming CFD simulations. We achieve almost four orders of speed-up with a much cheaper computational resource.

XB-SIM∗: A simulation framework for modeling and exploration of ReRAM-based CNN acceleration design

Resistive Random Access Memory (ReRAM)-based neural network accelerators have potential to surpass their digital counterparts in computational efficiency and performance. However, design of these accelerators faces a number of challenges including imperfections of the ReRAM device and a large amount of calculations required to accurately simulate the former. We present XB-SIM*, a simulation framework for ReRAM-crossbar-based Convolutional Neural Network (CNN) accelerators. XB-SIM* can be flexibly configured to simulate the accelerator's structure and clock-driven behaviors at the architecture level. This framework also includes an ReRAM-aware Neural Network (NN) training algorithm and a CNN-oriented mapper to train an NN and map it onto the simulated design efficiently. Behavior of the simulator has been verified by the corresponding circuit simulation of a real chip. Furthermore, a batch processing mode of the massive calculations that are required to mimic the behavior of ReRAM-crossbar circuits is proposed to fully apply the computational concurrency of the mapping strategy. On CPU/GPGPU, this batch processing mode can improve the simulation speed by up to 5.02× or 34.29×. Within this framework, comprehensive architectural exploration and end-to-end evaluation have been achieved, which provide some insights for systemic optimization.

MAISE: Construction of neural network interatomic models and evolutionary structure optimization

Module for ab initio structure evolution (MAISE) is an open-source package for materials modeling and prediction. The code’s main feature is an automated generation of neural network (NN) interatomic potentials for use in global structure searches. The systematic construction of Behler–Parrinello-type NN models approximating ab initio energy and forces relies on two approaches introduced in our recent studies. An evolutionary sampling scheme for generating reference structures improves the NNs’ mapping of regions visited in unconstrained searches, while a stratified training approach enables the creation of standardized NN models for multiple elements. A more flexible NN architecture proposed here expands the applicability of the stratified scheme for an arbitrary number of elements. The full workflow in the NN development is managed with a customizable ‘MAISE-NET’ wrapper written in Python. The global structure optimization capability in MAISE is based on an evolutionary algorithm applicable for nanoparticles, films, and bulk crystals. A multitribe extension of the algorithm allows for an efficient simultaneous optimization of nanoparticles in a given size range. Implemented structure analysis functions include fingerprinting with radial distribution functions and finding space groups with the SPGLIB tool. This work overviews MAISE’s available features, constructed models, and confirmed predictions. Program summaryProgram Title: MAISECPC Library link to program files:https://doi.org/10.17632/vfzgt2gnsh.1Licensing provisions: GNU General Public License v3.0Programming language: CNature of problem: Construction of NN interatomic potentials suitable for evolutionary structure searches, molecular dynamics, phonon calculations, and other applications presents a host of challenges ranging from sampling relevant parts of vast configuration spaces to tuning multitudes of NN parameters.Solution method: Evolutionary data generation and modular NN training algorithms featured in the open-source parallelized MAISE package enable automated development of NN models for multiple chemical species. Customizable MAISE-NET wrapper streamlines all stages of the iterative process.Unusual features: NN training stratification allows one to build libraries of reusable models from the bottom up, starting from elements and proceeding to multielement chemical systems. A multitribe evolutionary algorithm improves the efficiency of ground state structure searches by simultaneously optimizing nanoparticles of different sizes and periodically exchanging best motifs between the tribes.

Modeling of a Novel Coaxial Ducted Fan Aerial Robot Combined with Corner Environment by Using Artificial Neural Network.

A novel coaxial ducted fan aerial robot with a manipulator is proposed which can achieve some hover operation tasks in a corner environment, such as switching on and off a wall-attached button on the corner. In order to study the aerodynamic interference between the prototype and the environment when the aerial robot is hovering in the corner environment, a method for the comprehensive modeling of the prototype and corner environment based on the artificial neural network is presented. By using the CFD simulation software, the flow field of the prototype at different positions with the corner effect is analyzed. After determining the input, output and structure of the neural network model, the Adam and gradient descent algorithms are selected as the neural network training algorithms, respectively. In addition, to optimize the initial weights and biases of the neural network model, the genetic algorithm is precisely used. The three-dimensional prediction surfaces generated by the three methods of the neural network, kriging surface and the polynomial fitting are compared. The results show that the neural network has high prediction accuracy, and can be applied to the comprehensive modeling of the prototype and the corner environment.

Optimization of a Stirling Engine by Variable-Step Simplified Conjugate-Gradient Method and Neural Network Training Algorithm

The present study develops a novel optimization method for designing a Stirling engine by combining a variable-step simplified conjugate gradient method (VSCGM) and a neural network training algorithm. As compared with existing gradient-based methods, like the conjugate gradient method (CGM) and simplified conjugate gradient method (SCGM), the VSCGM method is a further modified version presented in this study which allows the convergence speed to be greatly accelerated while the form of the objective function can still be defined flexibly. Through the automatic adjustment of the variable step size, the optimal design is reached more efficiently and accurately. Therefore, the VSCGM appears to be a potential and alternative tool in a variety of engineering applications. In this study, optimization of a low-temperature-differential gamma-type Stirling engine was attempted as a test case. The optimizer was trained by the neural network algorithm based on the training data provided from three-dimensional computational fluid dynamic (CFD) computation. The optimal design of the influential parameters of the Stirling engine is yielded efficiently. Results show that the indicated work and thermal efficiency are increased with the present approach by 102.93% and 5.24%, respectively. Robustness of the VSCGM is tested by giving different sets of initial guesses.

Modeling a Thermochemical Reactor of a Solar Refrigerator by BaCl2-NH3 Sorption Using Artificial Neural Networks and Mathematical Symmetry Groups

The aim of this work is to present a model for heat transfer, desorbed refrigerant, and pressure of an intermittent solar cooling system’s thermochemical reactor based on backpropagation neural networks and mathematical symmetry groups. In order to achieve this, a reactor was designed and built based on the reaction of BaCl2-NH3. Experimental data from this reactor were collected, where barium chloride was used as a solid absorbent and ammonia as a refrigerant. The neural network was trained using the Levenberg–Marquardt algorithm. The correlation coefficient between experimental data and data simulated by the neural network was r = 0.9957. In the neural network’s sensitivity analysis, it was found that the inputs, reactor’s heating temperature and sorption time, influence neural network’s learning by 35% and 20%, respectively. It was also found that, by applying permutations to experimental data and using multibase mathematical symmetry groups, the neural network training algorithm converges faster.

Time series modeling of ionosphere total electron content using wavelet neural network and hybrid PSO training algorithm

در این مقاله از ترکیب شبکه‌های عصبی موجک سه‌لایه (WNNs) به همراه الگوریتم آموزش بروش بهنیه‌سازی انبوه ذرات هیبرید (PSO-BP) جهت مدل‌سازی تغییرات زمانی-مکانی محتوای الکترون کلی (TEC) یونوسفر در منطقه شمالغرب ایران (N-W) استفاده شده است. مشاهدات 30 روز ماه ژانویه سال 2018 جهت ارزیابی روش پیشنهادی بکار گرفته شده است. تعداد مشاهدات ورودی انتخاب شده جهت آموزش شبکه عصبی موجک سه‌لایه با الگوریتم آموزش PSO-BP بترتیب 20 و 10 ایستگاه از شبکه محلی آذربایجان می‌باشند. در هر 2 حالت تعداد 3 ایستگاه با توزیع مناسب به عنوان ایستگاه‌های آزمون در نظر گرفته شده‌اند. شاخص‌های آماری خطای نسبی، خطای |dVTEC|، انحراف معیار و ضریب همبستگی جهت ارزیابی مدل شبکه عصبی موجک مورد استفاده قرار گرفته است. نتایج حاصل از مدل پیشنهادی این مقاله با TEC حاصل از مشاهدات GPS به عنوان مرجع اصلی، مدل مرجع جهانی یونوسفر 2016 (IRI2016) و همچنین خروجی شبکه جهانی IGS (GIM) مقایسه شده است.

Cetane index prediction of ABE-diesel blends using empirical and artificial neural network models

ABSTRACT Recent developments in internal combustion engines have heightened the need for alternative biofuel. In the last five years, acetone-butanol-ethanol (ABE) has been extensively studied as a promising biofuel. However, the detailed investigation of its fuel properties has not been performed. One of the vital fuel properties is the cetane index. It is used to define the ignition quality of fuel, but its determination is painstaking and expensive. No previous study has utilized both empirical mathematical and ANN models to predict the cetane index of ABE-diesel blends. This study aims to predict ABE’s cetane index by comparing five empirical mathematical models with seven artificial neural networks (ANN) training algorithms. To the best of our knowledge, this is the first study to examine the cetane index of ABE-diesel blends using both empirical and ANN models. Results revealed that the feed-forward backpropagation network with 4 input, 10 hidden, and 1 output neurons that was trained with Levenberg-Marquardt algorithm (ANN-LM) showed the best performance with the highest values of R (0.9992) and R2 (0.9984). It also has the lowest values of MAD, MSE, RMSE and MAPE at 0.2572, 0.4456, 0.6675, and 0.5304, respectively. As compared to the best empirical mathematical model (the 3rd order polynomial), the ANN-LM had slightly better performance accuracy. Therefore, the 4–10-1 ANN structure trained with Levenberg-Marquardt was found to be the best predictor for cetane index of ABE-diesel blends at various blending ratios.

Mathematical simulation of the forecasting process of the fire hazard properties of substances

The article presents an original solution to the problem of fire hazard indicators of oil products determining using molecular descriptors and artificial neural networks. The modelling of the forecasting process was carried out on the basis of a neural network training algorithm with a teacher and an error back propagation algorithm. The resulting model makes it possible to determine the fire hazard properties of substances in real time without time span and other costs.

Forecast of export demand based on artificial neural network and fuzzy system theory

This paper analyses the significance and methods of foreign trade export forecasting. The index system of foreign trade export forecasting is determined based on the analysis of foreign trade export forecasting research results. The concepts and principles of artificial neural network and fuzzy system theory are expounded, and their respective advantages and disadvantages as well as their complementarities are analyzed. This paper introduces the types and training algorithms of evolutionary morphological neural network, combines the neural network with the fuzzy system theory, and establishes the prediction model. Finally, the evolutionary morphological neural network model is applied to the prediction of foreign trade export in view of the characteristics of export and considering the influence of various factors, the whole process of establishing evolutionary morphological neural network forecasting model is introduced in detail, and the change range of export is predicted, and the ideal forecasting results are obtained.

Stability Analysis of the Modified Levenberg-Marquardt Algorithm for the Artificial Neural Network Training.

The Levenberg-Marquardt and Newton are two algorithms that use the Hessian for the artificial neural network learning. In this article, we propose a modified Levenberg-Marquardt algorithm for the artificial neural network learning containing the training and testing stages. The modified Levenberg-Marquardt algorithm is based on the Levenberg-Marquardt and Newton algorithms but with the following two differences to assure the error stability and weights boundedness: 1) there is a singularity point in the learning rates of the Levenberg-Marquardt and Newton algorithms, while there is not a singularity point in the learning rate of the modified Levenberg-Marquardt algorithm and 2) the Levenberg-Marquardt and Newton algorithms have three different learning rates, while the modified Levenberg-Marquardt algorithm only has one learning rate. The error stability and weights boundedness of the modified Levenberg-Marquardt algorithm are assured based on the Lyapunov technique. We compare the artificial neural network learning with the modified Levenberg-Marquardt, Levenberg-Marquardt, Newton, and stable gradient algorithms for the learning of the electric and brain signals data set.

Comparative analysis of neural network training algorithms for the flood forecast modelling of an alluvial Himalayan river

AbstractThis study aimed to develop novel flood forecasting models using artificial neural network (ANN) algorithms. The models were assessed and validated for the case study of an alluvial river in the Kashmir region of the Indian Himalayas‐River Jhelum. In September 2014, a major flood occurred in the Kashmir Valley with peak discharge exceeding 115,000 m3/s; brought about by the multifarious interaction among atmospheric disturbances. The present study targeted the development of ANN models for flood prognosis. Thereby five neural network models were developed: Bayesian regularisation neural network, the Levenberg–Marquardt neural network, conjugate gradient neural network, scaled conjugate gradient neural network, and resilient back‐propagation neural network. Levenberg–Marquardt neural network model, with the mean squared error of 0.002128 (lowest of all models) and coefficient of determination of 95.839% (highest of all models), proved to be the best model based on the statistical validation parameters. All the simulations were evaluated and curves graphed for actual versus predicted discharges for 20% of the data. These models may be used to predict floods, and take advance precautionary measures by Irrigation and Flood Control Department of the State, rather than waiting for flood hydrograph to shoot (norm as of now).

Prediction of Maximum Flood Inundation Extents With Resilient Backpropagation Neural Network: Case Study of Kulmbach

In many countries, floods are the leading natural disaster in terms of damage and losses per year. Early prediction of such events can help prevent some of those losses. Artificial neural networks (ANN) show a strong ability to deal quickly with large amounts of measured data. In this work, we develop an ANN for outputting flood inundation maps based on multiple discharge inputs with a high grid resolution (4 m × 4 m). After testing different neural network training algorithms and network structures, we found resilience backpropagation to perform best. Furthermore, by introducing clustering for preprocessing discharge curves before training, the quality of the prediction could be improved. Synthetic flood events are used for the training and validation of the ANN. Historical events were additionally used for further validation with real data. The results show that the developed ANN is capable of predicting the maximum flood inundation extents. The mean squared error in more than 98 and 86% of the total area is smaller than 0.2 m2 in the prediction of synthetic events and historical events, respectively.

RESEARCH IN-DEPTH TRAINING OF NEURAL NETWORKS

Currently, deep learning of neural networks is one of the most popular methods for speech recognition, natural language processing, and computer vision. The article reviews the history of deep learning of neural networks and the current state in General. We consider algorithms for training neural networks used for deep training of neural networks, followed by fine-tuning using the method of back propagation of errors. Neural networks with large numbers of hidden layers, frequently occurring and disappearing gradients are very difficult to train. In this paper, we consider methods that successfully implement training of neural networks with large numbers of layers (more than one hundred) and vanishing gradients. A review of well-known libraries used for successful deep learning of neural networks is conducted.

Optimal Input Variables Disposition of Artificial Neural Networks Models for Enhancing Time Series Forecasting Accuracy

ABSTRACT Artificial Neural Networks (ANNs) models play an increasingly significant role in accurate time series prediction tools. However, an accurate time series forecasting using ANN requires an optimal model. Hence, great forecasting methods have been developed from optimized ANN models. Most of them focus more on input variables selection and preprocessing, topologies selection, optimum configuration and its associated parameters regardless of their input variables disposition. This paper provides an investigation of the effects of input variables disposition on ANNs models on training and forecasting performances. After investigation, a new ANNs optimization approach is proposed, consisting of finding optimal input variables disposition from the possible combinations. Therefore, a modified Back-Propagation neural networks training algorithm is presented in this paper. This proposed approach is applied to optimize the feed-forward and recurrent neural networks architectures; both built using traditional techniques, and pursuing to forecast the wind speed. Furthermore, the proposed approach is tested in a collaborative optimization method with single-objective optimization technique. Thus, Genetic Algorithm Back-Propagation neural networks aim to improve the forecasting accuracy relative to traditional methods was proposed. The experiment results demonstrate the requirement to take into consideration the input variables disposition to build a more optimal ANN model. They reveal that each proposed model is superior to its old considered model in terms of forecasting accuracy and thus show that the proposed optimization approach can be useful for time series forecasting accuracy improvement.

Mathematical Modeling of Photovoltaic Properties of Nipc/P-Si (Organic/Inorganic) Heterojunction by Using Artificial Neural Networks Model

In this research, the artificial neural network (ANN) and resilient back propagation (R-prop) training algorithm are utilized to model the photovoltaic properties of Nickel–phthalocyanine (NiPc/p-Si) heterojunction. The experimental data are extracted from experimental studies. Experimental data are utilized as inputs in the ANN model. Training of different structures of the ANN is processed to approach the minimum value of error. Eight artificial neural networks are trained to get a better mean square error (MSE) and best execution for the networks. The ANN performances are also investigated and their values are very small (MSE < 10-3). The simulation results of the current-voltage characteristics of NiPc films are produced and provided excellent matching with the corresponding experimental data. Utilization of ANN model for predictions is also processed and gives accurate results. The equation which describes the relation between the inputs and outputs is obtained. The high accuracy of the ANN model has appeared in the major guessing power and the ability of generalization depending on the obtained equations.