Artificial Neural Network Training Research Articles

A novel framework for effective structural vulnerability assessment of tubular structures using machine learning algorithms (GA and ANN) for hybrid simulations

Abstract Seismic vulnerability assessments are conventionally conducted by using sophisticated nonlinear analytical models, leading to aggressive computational demands. Previous attempts were made to reduce computational efforts for establishing vulnerability assessment of structures; however, the area of super tall and tubular structures still faces considerable lack. Advent of efficient machine learning (ML) has enabled engineering practitioners to automate the processes for fragility analysis; however, its application for high-rise tubular structures is not yet exploited, and most implementations are limited to basic ML. In this work, an attempt was made to reduce computational demand for the fragility assessment process for tubular structures by employing genetic algorithms (GAs) for nonlinear structural modeling, and development of artificial neural network (ANN) using deep learning for fragility development. Consequently, a simple lumped parameter model had been developed using open-source code of ZEUS-NL, containing parameters selected by GA to acutely account for convoluted interactive behavior of structural systems and dynamic demands. Subsequently, incremental dynamic analysis (IDA) was performed on the optimized model. A new framework has been established to develop and train ANN architecture by amalgamating Weka’s capability of data preprocessing with deep learning. The established ANN model resulted in correlation coefficient of 0.9972 and R 2 of 0.95, demonstrating adequate performance.

Artificial neural network model for water consumption prediction in dairy farms

This work presents a model based on artificial neural network (ANN) applied to predict water consumption in Brazilian dairy farms. Inputs were simple process data such as number of lactating cows, milk productivity, type of management, among others, with low computational cost and satisfactory data prediction. Data used for ANN training was acquired during two years from 31 farms in semi-confined dairy production. The analysis of the results was based on the following statistical models’ indicators: R2 (Coefficient of determination), BIAS (trend coefficient), MAE (mean absolute error), RMSE (Root-mean-square deviation), NRMSE (percentage of the mean of the observations) and RAE (Relative absolute error). After performing the ANN training, the results showed good accuracy to predict water consumption in Brazilian dairy farms, with an average absolute error of 28.4% being obtained. On the other hand, considering the dataset used for ANN validation, an average absolute error of 48% was obtained.

Changes in United States Summer Temperatures Revealed by Explainable Neural Networks

AbstractTo better understand the regional changes in summertime temperatures across the conterminous United States (CONUS), we adopt a recently developed machine learning framework that can be used to reveal the timing of emergence of forced climate signals from the noise of internal climate variability. Specifically, we train an artificial neural network (ANN) on seasonally averaged temperatures across the CONUS and then task the ANN to output the year associated with an individual map. In order to correctly identify the year, the ANN must therefore learn time‐evolving patterns of climate change amidst the noise of internal climate variability. The ANNs are first trained and tested on data from large ensembles and then evaluated using observations from a station‐based data set. To understand how the ANN is making its predictions, we leverage a collection of ad hoc feature attribution methods from explainable artificial intelligence (XAI). We find that anthropogenic signals in seasonal mean minimum temperature have emerged by the early 2000s for the CONUS, which occurred earliest in the Eastern United States. While our observational timing of emergence estimates are not as sensitive to the spatial resolution of the training data, we find a notable improvement in ANN skill using a higher resolution climate model, especially for its early twentieth century predictions. Composites of XAI maps reveal that this improvement is linked to temperatures around higher topography. We find that increases in spatial resolution of the ANN training data may yield benefits for machine learning applications in climate science.

Development of FSW Process Parameters for Lap Joints Made of Thin 7075 Aluminum Alloy Sheets.

The article describes machine learning using artificial neural networks (ANNs) to develop the parameters of the friction stir welding (FSW) process for three types of aluminum joints (EN AW 7075). The ANNs were built using a total of 608 experimental data. Two types of networks were built. The first one was used to classify good/bad joints with MLP 7-19-2 topology (one input layer with 7 neurons, one hidden layer with 19 neurons, and one output layer with 2 neurons), and the second one was used to regress the tensile load-bearing capacity with MLP 7-19-1 topology (one input layer with 7 neurons, one hidden layer with 19 neurons, and one output layer with 1 neuron). FSW parameters, such as rotational speed, welding speed, and joint and tool geometry, were used as input data for ANN training. The quality of the FSW joint was assessed in terms of microstructure and mechanical properties based on a case study. The usefulness of both trained neural networks has been demonstrated. The quality of the validation set for the regression network was approximately 93.6%, while the errors for the confusion matrix of the test set never exceeded 6%. Only 184 epochs were needed to train the regression network. The quality of the validation set was approximately 87.1%. Predictive maps were developed and presented in the work, allowing for the selection of optimal parameters of the FSW process for three types of joints.

GlioPredictor: a deep learning model for identification of high-risk adult IDH-mutant glioma towards adjuvant treatment planning

Identification of isocitrate dehydrogenase (IDH)-mutant glioma patients at high risk of early progression is critical for radiotherapy treatment planning. Currently tools to stratify risk of early progression are lacking. We sought to identify a combination of molecular markers that could be used to identify patients who may have a greater need for adjuvant radiation therapy machine learning technology. 507 WHO Grade 2 and 3 glioma cases from The Cancer Genome Atlas, and 1309 cases from AACR GENIE v13.0 datasets were studied for genetic disparities between IDH1-wildtype and IDH1-mutant cohorts, and between different age groups. Genetic features such as mutations and copy number variations (CNVs) correlated with IDH1 mutation status were selected as potential inputs to train artificial neural networks (ANNs) to predict IDH1 mutation status. Grade 2 and 3 glioma cases from the Memorial Sloan Kettering dataset (n = 404) and Grade 3 glioma cases with subtotal resection (STR) from Northwestern University (NU) (n = 21) were used to further evaluate the best performing ANN model as independent datasets. IDH1 mutation is associated with decreased CNVs of EGFR (21% vs. 3%), CDKN2A (20% vs. 6%), PTEN (14% vs. 1.7%), and increased percentage of mutations for TP53 (15% vs. 63%), and ATRX (10% vs. 54%), which were all statistically significant (p < 0.001). Age > 40 was unable to identify high-risk IDH1-mutant with early progression. A glioma early progression risk prediction (GlioPredictor) score generated from the best performing ANN model (6/6/6/6/2/1) with 6 inputs, including CNVs of EGFR, PTEN and CDKN2A, mutation status of TP53 and ATRX, patient’s age can predict IDH1 mutation status with over 90% accuracy. The GlioPredictor score identified a subgroup of high-risk IDH1-mutant in TCGA and NU datasets with early disease progression (p = 0.0019, 0.0238, respectively). The GlioPredictor that integrates age at diagnosis, CNVs of EGFR, CDKN2A, PTEN and mutation status of TP53, and ATRX can identify a small cohort of IDH-mutant with high risk of early progression. The current version of GlioPredictor mainly incorporated clinically often tested genetic biomarkers. Considering complexity of clinical and genetic features that correlate with glioma progression, future derivatives of GlioPredictor incorporating more inputs can be a potential supplement for adjuvant radiotherapy patient selection of IDH-mutant glioma patients.

Switched Reluctance Machine Modeling through Multilayer Neural Networks

The work deals with the application of artificial neural networks (ANNs) in the modeling of switched reluctance machines (SRMs). The performance of a SRM is determined by its geometry, materials used and levels of excitation. In this way, this work investigates the influence of the stator and rotor back iron thickness in the performance of SRM. A multilayer neural network is proposed to learn the nonlinear characteristics of the motor. Data of flux linkages and torque are obtained through simulations of finite elements and used for ANN training. The algorithm developed in Octave allows the user to adjust the network parameters. The results presented confirm the feasibility of using ANN to establish a predictive model of SRM performance, thus enabling further investigation in the future.

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks

The use of mini-batches of data in training artificial neural networks is nowadays very common. Despite its broad usage, theories explaining quantitatively how large or small the optimal mini-batch size should be are missing. This work presents a systematic attempt at understanding the role of the mini-batch size in training two-layer neural networks. Working in the teacher-student scenario, with a sparse teacher, and focusing on tasks of different complexity, we quantify the effects of changing the mini-batch size m. We find that often the generalization performances of the student strongly depend on m and may undergo sharp phase transitions at a critical value mc , such that for m<mc the training process fails, while for m>mc the student learns perfectly or generalizes very well the teacher. Phase transitions are induced by collective phenomena firstly discovered in statistical mechanics and later observed in many fields of science. Observing a phase transition by varying the mini-batch size across different architectures raises several questions about the role of this hyperparameter in the neural network learning process.

Controlled descent training

SummaryIn this work, a novel and model‐based artificial neural network (ANN) training method is developed supported by optimal control theory. The method augments training labels in order to robustly guarantee training loss convergence and improve training convergence rate. Dynamic label augmentation is proposed within the framework of gradient descent training where the convergence of training loss is controlled. First, we capture the training behavior with the help of empirical Neural Tangent Kernels (NTK) and borrow tools from systems and control theory to analyze both the local and global training dynamics (e.g., stability, reachability). Second, we propose to dynamically alter the gradient descent training mechanism via fictitious labels as control inputs and an optimal state feedback policy. In this way, we enforce locally optimal and convergent training behavior. The novel algorithm, Controlled Descent Training (CDT), guarantees local convergence. CDT unleashes new potentials in the analysis, interpretation, and design of ANN architectures. The applicability of the method is demonstrated on standard regression and classification problems.

Artificial intelligence-based speckle featurization and localization for ultrasound speckle tracking velocimetry

Deep learning-based super-resolution ultrasound (DL-SRU) framework has been successful in improving spatial resolution and measuring the velocity field information of a blood flows by localizing and tracking speckle signals of red blood cells (RBCs) without using any contrast agents. However, DL-SRU can localize only a small part of the speckle signals of blood flow owing to ambiguity problems encountered in the classification of blood flow signals from ultrasound B-mode images and the building up of suitable datasets required for training artificial neural networks, as well as the structural limitations of the neural network itself. An artificial intelligence-based speckle featurization and localization (AI-SFL) framework is proposed in this study. It includes a machine learning-based algorithm for classifying blood flow signals from ultrasound B-mode images, dimensionality reduction for featurizing speckle patterns of the classified blood flow signals by approximating them with quantitative values. A novel and robust neural network (ResSU-net) is trained using the online data generation (ODG) method and the extracted speckle features. The super-resolution performance of the proposed AI-SFL and ODG method is evaluated and compared with the results of previous U-net and conventional data augmentation methods under in silico conditions. The predicted locations of RBCs by the AI-SFL and DL-SRU for speckle patterns of blood flow are applied to a PTV algorithm to measure quantitative velocity fields of the flow. Finally, the feasibility of the proposed AI-SFL framework for measuring real blood flows is verified under in vivo conditions.

An application of a genetic algorithm in co-optimization of geological CO2 storage based on artificial neural networks

Abstract Global warming, driven by human-induced disruptions to the natural carbon dioxide (CO2) cycle, is a pressing concern. To mitigate this, carbon capture and storage has emerged as a key strategy that enables the continued use of fossil fuels while transitioning to cleaner energy sources. Deep saline aquifers are of particular interest due to their substantial CO2 storage potential, often located near fossil fuel reservoirs. In this study, a deep saline aquifer model with a saline water production well was constructed to develop the optimization workflow. Due to the time-consuming nature of each realization of the numerical simulation, we introduce a surrogate aquifer model derived from extracted data. The novelty of our work lies in the pioneering of simultaneous optimization using machine learning within an integrated framework. Unlike previous studies, which typically focused on single-parameter optimization, our research addresses this gap by performing multi-objective optimization for CO2 storage and breakthrough time in deep saline aquifers using a data-driven model. Our methodology encompasses preprocessing and feature selection, identifying eight pivotal parameters. Evaluation metrics include root mean square error (RMSE), mean absolute percentage error (MAPE) and R2. In predicting CO2 storage values, RMSE, MAPE and R2 in test data were 2.07%, 1.52% and 0.99, respectively, while in blind data, they were 2.5%, 2.05% and 0.99. For the CO2 breakthrough time, RMSE, MAPE and R2 in the test data were 2.1%, 1.77% and 0.93, while in the blind data they were 2.8%, 2.23% and 0.92, respectively. In addressing the substantial computational demands and time-consuming nature of coupling a numerical simulator with an optimization algorithm, we have adopted a strategy in which the trained artificial neural network is seamlessly integrated with a multi-objective genetic algorithm. Within this framework, we conducted 5000 comprehensive experiments to rigorously validate the development of the Pareto front, highlighting the depth of our computational approach. The findings of the study promise insights into the interplay between CO2 breakthrough time and storage in aquifer-based carbon capture and storage processes within an integrated framework based on data-driven coupled multi-objective optimization.

Prediction of 30-Day Morbidity and Mortality After Conversion of Sleeve Gastrectomy to Roux-en-Y Gastric Bypass: Use of an Artificial Neural Network.

Conversion of sleeve gastrectomy to Roux-en-Y gastric bypass is indicated primarily for unsatisfactory weight loss or gastroesophageal reflux disease (GERD). This study aimed to use a comprehensive database to define predictors of 30-day reoperation, readmission, reintervention, or mortality. An artificial neural network (ANN) was employed to optimize prediction of the composite endpoint (occurrence of 1+ morbid event). Areview of 8895 patients who underwent conversion for weight-related or GERD-related indications was performed using the 2021 MBSAQIP national dataset. Demographics, comorbidities, laboratory values, and other factors were assessed for bivariate and subsequent multivariable associations with the composite endpoint (P ≤ .05). Factors considered in the multivariable model were imputed into a three-node ANN with 20% randomly withheld for internal validation, to optimize predictive accuracy. Models were compared using receiver operating characteristic (ROC) curve analysis. 39% underwent conversion for weight considerations and 61% for GERD. Rates of 30-day reoperation, readmission, reintervention, mortality, and the composite endpoint were 3.0%, 7.1%, 2.1%, .1%, and 9.1%, respectively. Of the nine factors associated with the composite endpoint on bivariate analysis, only non-white race (P < .001; odds ratio 1.4), lower body-mass index (P < .001; odds ratio .22), and therapeutic anticoagulation (P = .001; odds ratio 2.0) remained significant upon multivariable analysis. Areas under ROC curves for the multivariable regression, ANN training, and validation sets were .587, .601, and .604, respectively. Identification of risk factors for morbidity after conversion offers critical information to improve patient selection and manage postoperative expectations. ANN models, with appropriate clinical integration, may optimize prediction of morbidity.

Prediction of ball-on-plate friction and wear by ANN with data-driven optimization

For training artificial neural network (ANN), big data either generated by machine or measured from experiments are used as input to “learn” the unspecified functions defining the ANN. The experimental data are fed directly into the optimizer allowing training to be performed according to a predefined loss function. To predict sliding friction and wear at mixed lubrication conditions, in this study a specific ANN structure was so designed that deep learning algorithms and data-driven optimization models can be used. Experimental ball-on-plate friction and wear data were analyzed using the specific training procedure to optimize the weights and biases incorporated into the neural layers of the ANN, and only two independent experimental data sets were used during the ANN optimization procedure. After the training procedure, the ANN is capable to predict the contact and hydrodynamic pressure by adapting the output data according to the tribological condition implemented in the optimization algorithm.

Optimization of GFRP-concrete-steel composite column based on genetic algorithm - artificial neural network

Genetic algorithms (GA) and artificial neural networks (ANN) as intelligent algorithms have been widely applied in engineering fields such as structural design. The GFRP-concrete-steel composite column (GCS), known for its corrosion resistance, has garnered significant attention in its design. However, the application of GA and ANN methods in the context of GCS has primarily been confined to enhancing the predictive accuracy of ANN. This paper introduces a novel approach for the structural optimization of GCS using GA and ANN. A total of 1050 finite element models were generated and analyzed using ANSYS for the purpose of training ANN. The validity of these finite element models was confirmed through comparative analysis with experimental data. The trained ANN was then utilized to predict the loading capacity of GCS for each generation in the GA. By collecting pricing data for GFRP tubes, steel tubes, and concrete under various parameters, pricing fitting formulas for the three materials were derived. Subsequently, the cost of GCS under different parameter sets was calculated. The maximization of the ratio of GCS loading capacity to its cost (LP) was established as the optimization objective. Through the iterative optimization process of the GA-ANN algorithm, the structural optimization was achieved to maximize the economic efficiency of GCS. Comparing the optimization results with numerical simulation results showed that errors were mostly contained within 10 % or even as low as 5 %, thereby validating the accuracy of the GA-ANN algorithm. The applicability of the GA-ANN algorithm to GCS structural optimization was demonstrated by altering a single concrete variable. The method proposed in this paper can be effectively applied to the structural design and optimization of composite columns with a sandwich structure, such as GCS.

Voltage and Overpotential Prediction of Vanadium Redox Flow Batteries with Artificial Neural Networks

This article explores the novel application of a trained artificial neural network (ANN) in the prediction of vanadium redox flow battery behaviour and compares its performance with that of a two-dimensional numerical model. The aim is to evaluate the capability of two ANNs, one for predicting the cell potential and one for the overpotential under various operating conditions. The two-dimensional model, previously validated with experimental data, was used to generate data to train and test the ANNs. The results show that the first ANN precisely predicts the cell voltage under different states of charge and current density conditions in both the charge and discharge modes. The second ANN, which is responsible for the overpotential calculation, can accurately predict the overpotential across the cell domains, with the lowest confidence near high-gradient areas such as the electrode membrane and domain boundaries. Furthermore, the computational time is substantially reduced, making ANNs a suitable option for the fast understanding and optimisation of VRFBs.

Chip-In-Loop SNN Proxy Learning: a new method for efficient training of spiking neural networks.

The primary approaches used to train spiking neural networks (SNNs) involve either training artificial neural networks (ANNs) first and then transforming them into SNNs, or directly training SNNs using surrogate gradient techniques. Nevertheless, both of these methods encounter a shared challenge: they rely on frame-based methodologies, where asynchronous events are gathered into synchronous frames for computation. This strays from the authentic asynchronous, event-driven nature of SNNs, resulting in notable performance degradation when deploying the trained models on SNN simulators or hardware chips for real-time asynchronous computation. To eliminate this performance degradation, we propose a hardware-based SNN proxy learning method that is called Chip-In-Loop SNN Proxy Learning (CIL-SPL). This approach effectively eliminates the performance degradation caused by the mismatch between synchronous and asynchronous computations. To demonstrate the effectiveness of our method, we trained models using public datasets such as N-MNIST and tested them on the SNN simulator or hardware chip, comparing our results to those classical training methods.

PARALLEL AND DISTRIBUTED COMPUTING TECHNOLOGIES FOR AUTONOMOUS VEHICLE NAVIGATION

Context. Autonomous vehicles are becoming increasingly popular, and one of the important modern challenges in their development is ensuring their effective navigation in space and movement within designated lanes. This paper examines a method of spatial orientation for vehicles using computer vision and artificial neural networks. The research focused on the navigation system of an autonomous vehicle, which incorporates the use of modern distributed and parallel computing technologies.&#x0D; Objective. The aim of this work is to enhance modern autonomous vehicle navigation algorithms through parallel training of artificial neural networks and to determine the optimal combination of technologies and nodes of devices to increase speed and enable real-time decision-making capabilities in spatial navigation for autonomous vehicles.&#x0D; Method. The research establishes that the utilization of computer vision and neural networks for road lane segmentation proves to be an effective method for spatial orientation of autonomous vehicles. For multi-core computing systems, the application of parallel programming technology, OpenMP, for neural network training on processors with varying numbers of parallel threads increases the algorithm’s execution speed. However, the use of CUDA technology for neural network training on a graphics processing unit significantly enhances prediction speeds compared to OpenMP. Additionally, the feasibility of employing PyTorch Distributed Data Parallel (DDP) technology for training the neural network across multiple graphics processing units (nodes) simultaneously was explored. This approach further improved prediction execution times compared to using a single graphics processing unit.&#x0D; Results. An algorithm for training and prediction of an artificial neural network was developed using two independent nodes, each equipped with separate graphics processing units, and their synchronization for exchanging training results after each epoch, employing PyTorch Distributed Data Parallel (DDP) technology. This approach allows for scalable computations across a higher number of resources, significantly expediting the model training process.&#x0D; Conclusions. The conducted experiments have affirmed the effectiveness of the proposed algorithm, warranting the recommendation of this research for further advancement in autonomous vehicles and enhancement of their navigational capabilities. Notably, the research outcomes can find applications in various domains, encompassing automotive manufacturing, logistics, and urban transportation infrastructure. The obtained results are expected to assist future researchers in understanding the most efficient hardware and software resources to employ for implementing AI-based navigation systems in autonomous vehicles. Prospects for future investigations may encompass refining the accuracy of the proposed parallel algorithm without compromising its efficiency metrics. Furthermore, there is potential for experimental exploration of the proposed algorithm in more intricate practical scenarios of diverse nature and dimensions.

Rapid and accurate predictions of perfect and defective material properties in atomistic simulation using the power of 3D CNN-based trained artificial neural networks

This article introduces an innovative approach that utilizes machine learning (ML) to address the computational challenges of accurate atomistic simulations in materials science. Focusing on the field of molecular dynamics (MD), which offers insight into material behavior at the atomic level, the study demonstrates the potential of trained artificial neural networks (tANNs) as surrogate models. These tANNs capture complex patterns from built datasets, enabling fast and accurate predictions of material properties. The article highlights the application of 3D convolutional neural networks (CNNs) to incorporate atomistic details and defects in predictions, a significant advancement compared to current 2D image-based, or descriptor-based methods. Through a dataset of atomistic structures and MD simulations, the trained 3D CNN achieves impressive accuracy, predicting material properties with a root-mean-square error below 0.65 GPa for the prediction of elastic constants and a speed-up of approximately 185 to 2100 times compared to traditional MD simulations. This breakthrough promises to expedite materials design processes and facilitate scale-bridging in materials science, offering a new perspective on addressing computational demands in atomistic simulations.

Desing of VLSI Architecture for a flexible testbed of Artificial Neural Network for training and testing on FPGA

Desing of VLSI Architecture for a flexible testbed of Artificial Neural Network for training and testing on FPGA

Определение модуля деформации бинарного композита с помощью искусственной нейронной сети

Using of existing methods of determining the characteristics of soils which are part of current regulatory documents and which are based on the hypothesis of normal character of distribution require considerable time and material costs. According to the results of conducted laboratory researches the hypothesis wasn’t confirmed. In the paper it proposes to use trained artificial neural network for determination of the deformation modulus of binary composite «sand — granules of expanded polystyrene». Thus, it has been proven efficiency proposing method using trained artificial neural network in compare classical regression equation for determination of the deformation modulus of the binary composite. With a confidence probability of P = 95 % the absolute value of the relative error is equal to 11,8 % the proposing learning artificial neural network in 11 times less than the absolute value of the relative error of classical regression equation. Also with a confidence probability of P = 95 % the coefficient of determination is equal to 0,5641 and in 6,6 times less than it of regression equation. Further research will be directed to the selection of the values of the parameters of the artificial neural network program for increase the accuracy of determining the deformation modulus of the binary composite.

Azimuth estimation based on CNN and LSTM for geomagnetic and inertial sensors data

Although estimating the azimuth using a geomagnetic sensor is very useful, the estimation error may be very large due to the surrounding geomagnetic disturbance. We proposed a novel method for preprocessing appropriately for geomagnetic and inertial sensor data to be suitable for the proposed Artificial Neural Network model and training method for the model. As a result, the probability of azimuth estimation error within 1 degree is 96.4% with regression estimation. For classification estimation, when the azimuth estimation probability is 90% or more, the probability that the azimuth estimation error is within 1 degree is 100%.