Artificial Neural Network Training Research Articles

Quantifying entanglement for unknown quantum states via artificial neural networks

Quantum entanglement acts as a crucial part in quantum computation and quantum information, hence quantifying unknown entanglement is an important task. Due to the fact that the amount of entanglement cannot be achieved directly by measuring any physical observables, it remains an open problem to quantify entanglement experimentally. In this work, we provide an effective way to quantify entanglement for the unknown quantum states via artificial neural networks. By choosing the expectation values of measurements as input features and the values of entanglement measures as labels, we train artificial neural network models to predict the entanglement for new quantum states accurately. Our method does not require the full information about unknown quantum states, which highlights the effectiveness and versatility of machine learning in exploring quantum entanglement.

Reinforcement Learning for EV Fleet Smart Charging with On-Site Renewable Energy Sources

In 2020, the transportation sector was the second largest source of carbon emissions in the UK and in Newcastle upon Tyne, responsible for about 33% of total emissions. To support the UK’s target of reaching net zero emissions by 2050, electric vehicles (EVs) are pivotal in advancing carbon-neutral road transportation. Optimal EV charging requires a better understanding of the unpredictable output from on-site renewable energy sources (ORES). This paper proposes an integrated EV fleet charging schedule with a proximal policy optimization method based on a framework for deep reinforcement learning. For the design of the reinforcement learning environment, mathematical models of wind and solar power generation are created. In addition, the multivariate Gaussian distributions derived from historical weather and EV fleet charging data are utilized to simulate weather and charging demand uncertainty in order to create large datasets for training the model. The optimization problem is expressed as a Markov decision process (MDP) with operational constraints. For training artificial neural networks (ANNs) through successive transition simulations, a proximal policy optimization (PPO) approach is devised. The optimization approach is deployed and evaluated on a real-world scenario comprised of council EV fleet charging data from Leicester, UK. The results show that due to the design of the rewards function and system limitations, the charging action is biased towards the time of day when renewable energy output is maximum (midday). The charging decision by reinforcement learning improves the utilization of renewable energy by 2–4% compared to the random charging policy and the priority charging policy. This study contributes to the reduction in battery charging and discharging, electricity sold to the grid to create benefits and the reduction in carbon emissions.

Robust chemical analysis with graphene chemosensors and machine learning.

Ion-sensitive field-effect transistors (ISFETs) have emerged as indispensable tools in chemosensing applications1-4. ISFETs operate by converting changes in the composition of chemical solutions into electrical signals, making them ideal for environmental monitoring5,6, healthcare diagnostics7 and industrial process control8. Recent advancements in ISFET technology, including functionalized multiplexed arrays and advanced data analytics, have improved their performance9,10. Here we illustrate the advantages of incorporating machine learning algorithms to construct predictive models using the extensive datasets generated by ISFET sensors for both classification and quantification tasks. This integration also sheds new light on the working of ISFETs beyond what can be derived solely from human expertise. Furthermore, it mitigates practical challenges associated with cycle-to-cycle, sensor-to-sensor and chip-to-chip variations, paving the way for the broader adoption of ISFETs in commercial applications. Specifically, we use data generated bynon-functionalized graphene-based ISFET arrays to train artificial neural networks that possess a remarkable ability to discern instances of food fraud, food spoilage and food safety concerns. We anticipate that the fusion of compact, energy-efficient and reusable graphene-based ISFET technology with robust machine learning algorithms holds the potential to revolutionize the detection of subtle chemical and environmental changes, offering swift, data-driven insights applicable across a wide spectrum of applications.

Network anomaly detection using Deep Autoencoder and parallel Artificial Bee Colony algorithm-trained neural network

Cyberattacks are increasingly becoming more complex, which makes intrusion detection extremely difficult. Several intrusion detection approaches have been developed in the literature and utilized to tackle computer security intrusions. Implementing machine learning and deep learning models for network intrusion detection has been a topic of active research in cybersecurity. In this study, artificial neural networks (ANNs), a type of machine learning algorithm, are employed to determine optimal network weight sets during the training phase. Conventional training algorithms, such as back-propagation, may encounter challenges in optimization due to being entrapped within local minima during the iterative optimization process; global search strategies can be slow at locating global minima, and they may suffer from a low detection rate. In the ANN training, the Artificial Bee Colony (ABC) algorithm enables the avoidance of local minimum solutions by conducting a high-performance search in the solution space but it needs some modifications. To address these challenges, this work suggests a Deep Autoencoder (DAE)-based, vectorized, and parallelized ABC algorithm for training feed-forward artificial neural networks, which is tested on the UNSW-NB15 and NF-UNSW-NB15-v2 datasets. Our experimental results demonstrate that the proposed DAE-based parallel ABC-ANN outperforms existing metaheuristics, showing notable improvements in network intrusion detection. The experimental results reveal a notable improvement in network intrusion detection through this proposed approach, exhibiting an increase in detection rate (DR) by 0.76 to 0.81 and a reduction in false alarm rate (FAR) by 0.016 to 0.005 compared to the ANN-BP algorithm on the UNSW-NB15 dataset. Furthermore, there is a reduction in FAR by 0.006 to 0.0003 compared to the ANN-BP algorithm on the NF-UNSW-NB15-v2 dataset. These findings underscore the effectiveness of our proposed approach in enhancing network security against network intrusions.

Predicting and analysis of urban wastewater quantity to be treated by using Neural Network and ODE model: A case study

Wastewater data, including Population of the Municipality Served by the Wastewater Treatment Plants (WWTPs), Capacity, Number of WWTPs, Amount of Daily Wastewater Discharged per Person and Wastewater Treated in WWTPs (y(t)) obtained from Turkish Statistical Institute (TUIK) for Kayseri province for 2003-2020 were mathematically modelled and analysed with Artificial Neural Network (ANN) and Differential equations. Firstly, the data were augmented through ARIMA since TUIK data numbers are insufficient to ANN training and later the data were normalized. The augmented normalized data was trained with ANN twice, thus the effect of other variables on the y(t) variable was shown and mathematical ANN activation functions in the form of a tangent hyperbolic function was proposed for this variable. Then, arbitrary parameters used for a linear system consisting of differential equations representing the five variables mentioned above were estimated using normalized original data and thus the ODE (ordinary differential equation) model was proposed. Two ANN models and ODE model were evaluated on normalized real TUIK data and the performances of these three models were compared. Among these mathematical models, the model that gave the minimum MSE (mean squared error) has been determined as the ODE model. Finally, future predictions were made for the y(t) variable with the ODE model.

A bivariate dependent degradation model based on artificial neural network supported stochastic process and Copula function

AbstractIn order to use the high ability of the artificial neural network (ANN) in data fitting, this paper introduces an ANN in stochastic process to describe the mean function for degradation modeling. Due to the fact that the existing method cannot handle the bivariate dependent degradation conditions, a bivariate dependent degradation model based on Copula function and ANN‐supported stochastic processes is proposed. Considering the random effects caused by individual difference, it is assumed that the unknown parameters in the stochastic processes and Copula functions are randomly distributed. Based on the maximum likelihood and moment estimation methods, a related statistical inference method for ANN training and parameter estimation is developed to use the bivariate dependent degradation model. An actual fatigue crack dataset is used to demonstrate the validity of the proposed method. The obtained results show that the dependent relationship between two degradation indicators should not be neglected, and it can be efficiently handled by the proposed method. Furthermore, the proposed degradation model can provide reliability and degradation intervals with enough precision due to the fact that it considers the random effects caused by individual difference.

Removal of zinc from wastewaters using Turkish bentonite and artificial neural network [ANN] modeling

In this study, Ordu-Unye bentonite was used as an adsorbent in the removal of zinc from aqueous solutions. The aim of the experimental part of the study was to ascertain how zinc removal was affected by variables such as pH, adsorbent amount, contact time, and initial zinc concentration. In the second part of the experiments, bentonite was modified with two different acids and the adsorption performance of modified bentonite was also investigated. Characterization of raw and modified bentonites was also carried out using FTIR and XRD. It was observed that acid modification of bentonite negatively affected the zinc removal process from aqueous solutions. In this study, higher zinc removal (95 %) was obtained with raw bentonite compared to acid modified bentonites (58.4 % in HNO3 activated, 43.8 % for H2SO4 activated). Equilibrium isotherms were obtained and modelled to explain the adsorption mechanism. Adsorption isotherm studies showed that zinc adsorption fits well with Langmuir (R2: 0.99) and Temkin (R2: 0.97) models. Besides from these experimental investigations, various artificial neural network (ANN) training techniques were used to optimize the zinc adsorption process. By trial and error, the optimal performance was obtained by changing the number of hidden neurons in each layer of the neural network architecture. These models under study were analyzed to determine their R2 and mean square error (MSE) values, and the optimal outcomes were identified. Among the various training models of ANN, it was determined that the Bayesian Regularization method exhibited the optimum network architecture with the highest R2 (R2:0.995) and lowest MSE (MSE:0.0008) ratio.

Mechanism-based and data-driven approach to developing the constitutive model of viscoelastic elastomers

Constitutive modeling of viscoelastic elastomers has been an active field for decades. In this work, we develop a mechanism-based and data-driven method to develop constitutive models of viscoelastic elastomers under large deformation. Based on the theory of finite deformation viscoelasticity, the feature of strain energy density function is utilized when we design the machine learning architecture, which allows for fast generation of qualified artificial data to train artificial neural networks (ANNs). According to the typical microstructures of elastomers, three groups of ANNs are established to determine the strain energy density functions of the hyperelastic and viscous polymer networks, which are further tested by experimental data of our own and those in the literature. The machine learning architecture also allows for flexible expansion of the ANN database to consider newly-developed elastomers. The developed constitutive model of the material automatically satisfies the laws of thermodynamics and can be easily implemented in finite element analysis for more complex structures and loading conditions. The developed numerical and experimental framework provides an efficient paradigm for constitutive modeling of viscoelastic elastomers.

Predictive residual stress analysis in gear units: an artificial intelligence approach based on finite element data

Determining residual stresses is essential for ensuring the safety and longevity of gear components. Traditional methods, such as analytical, numerical, and experimental approaches, are often costly, time-consuming, and sometimes destructive. This study proposes a new method to predict residual stresses in gear units using artificial intelligence based on data from finite element analysis. To achieve this, a commercial finite element tool models the heat treatment and carburization processes, grounded in thermomechanical metallurgical physics. The residual stresses obtained from finite element analysis are then compared to the ones from experiment using deep hole drilling approach. This is followed by a sensitivity analysis. Next, a specific class of gearbox component geometries and the computed stress fields are utilized to train artificial neural networks. The performance of the artificial neural networks approach is compared to that of two different machine learning regression methods. It has been concluded that this technique successfully anticipates residual stresses in components with scaled geometries and similar heat treatment steps, outperforming the two regression models in accuracy and efficiency. Importantly, it achieves this with incomparable, significantly less computation time compared to standard finite element analysis.

Engineering flexible machine learning systems by traversing functionally invariant paths

Contemporary machine learning algorithms train artificial neural networks by setting network weights to a single optimized configuration through gradient descent on task-specific training data. The resulting networks can achieve human-level performance on natural language processing, image analysis and agent-based tasks, but lack the flexibility and robustness characteristic of human intelligence. Here we introduce a differential geometry framework—functionally invariant paths—that provides flexible and continuous adaptation of trained neural networks so that secondary tasks can be achieved beyond the main machine learning goal, including increased network sparsification and adversarial robustness. We formulate the weight space of a neural network as a curved Riemannian manifold equipped with a metric tensor whose spectrum defines low-rank subspaces in weight space that accommodate network adaptation without loss of prior knowledge. We formalize adaptation as movement along a geodesic path in weight space while searching for networks that accommodate secondary objectives. With modest computational resources, the functionally invariant path algorithm achieves performance comparable with or exceeding state-of-the-art methods including low-rank adaptation on continual learning, sparsification and adversarial robustness tasks for large language models (bidirectional encoder representations from transformers), vision transformers (ViT and DeIT) and convolutional neural networks.

Training artificial neural networks using self-organizing migrating algorithm for skin segmentation.

This study presents an application of the self-organizing migrating algorithm (SOMA) to train artificial neural networks for skin segmentation tasks. We compare the performance of SOMA with popular gradient-based optimization methods such as ADAM and SGDM, as well as with another evolutionary algorithm, differential evolution (DE). Experiments are conducted on the skin dataset, which consists of 245,057 samples with skin and non-skin labels. The results show that the neural network trained by SOMA achieves the highest accuracy (93.18%), outperforming ADAM (84.87%), SGDM (84.79%), and DE (91.32%). The visual evaluation also reveals the SOMA-trained neural network's accurate and reliable segmentation capabilities in most cases. These findings highlight the potential of incorporating evolutionary optimization algorithms like SOMA into the training process of artificial neural networks, significantly improving performance in image segmentation tasks.

The prediction of bolted joint loosening state under transverse load based on a data-driven model

The impact of transverse load results in the loosening of bolted joints. The loosening condition is determined based on the transverse load and relevant parameters of the bolted joint. To efficiently forecast the loosening state of bolts under various transverse loads and relevant parameters, this study proposes a predictive model for the loosening state of bolted joints. Initially, the relationship between the loosening criteria and relevant parameters is established according to the bolt’s force conditions. Through elementary effect analysis, the high-dimensional representation model is improved, breaking down the loosening criteria function with high-dimensional parameter inputs into a series of low-dimensional functions. Subsequently, the prediction of the loosening state is achieved by training artificial neural networks to approximate multiple low-dimensional functions. The improved predictive model is utilized to determine the critical load for bolted joint loosening. Validation indicates the accuracy of the proposed model. Notably, the influence of the bearing friction coefficient and preload on the critical transverse load is more pronounced compared to the thread friction coefficient.

Parametric Analysis of Moment-Resisting Timber Frames Combined with Cross Laminated Timber Walls and Prediction Models Using Nonlinear Regression and Artificial Neural Networks

The light weight and moderate stiffness of multistorey timber buildings make them susceptible to increased lateral displacements and accelerations under service-level wind loading. Therefore, the fulfilment of serviceability requirements is a major challenge. In this study, linear elastic finite element analysis was used to perform a parametric study of moment-resisting timber frames combined with cross laminated timber walls. In the parametric study, various mechanical and geometrical parameters were varied within practical ranges. The results of the parametric study were used to derive simplified analytical expressions and to train artificial neural networks which can be used to estimate fundamental frequency, mode shape, top floor displacement, maximum inter-storey drift, and wind-induced acceleration. The analytical expressions and the artificial neural networks can be used for the preliminary assessment of serviceability performance of moment-resisting timber frames with and without cross laminated timber walls, under service-level wind loading.

A Comparative Study of Artificial Neural Network Training Algorithms to Predict Photovoltaic Module Output Power

Introduction: Solar energy is a crucial component and contributes a large portion of renewable resources, the demand for which has recently increased. During previous years, it was difficult to predict the amount of energy obtained from photovoltaic systems. The angle of inclination of the solar panel, the amount of radiation, the speed of the wind, the amount of humidity, and the temperature of the weather are major factors that effectively affect the production of electrical energy. Scientists have used many strategies to predict the power generated by PV modules accurately, but each method has different pros and cons. Method: This study tested three training algorithms for artificial neural networks (ANN): scaled conjugate gradient (SCG), Levenberg Marquardt (LM), and Bayesian regularization (BR), determining which one performed best in terms of prediction speed and accuracy. Twenty-eight thousand two hundred ninety-six samples of experimental data for primary influencing environmental factors were fed into the artificial neural network, which consists of 15 hidden layers. Before training the network, we preprocessed the data to remove factors that have a secondary effect. Result: The analytical results showed that the artificial neural network trained according to the LM algorithm is the best in terms of accuracy and speed of predicting the resulting photovoltaic energy. Conclusion: The results showed that although the regression evaluation and MSE values for the LM, SCG, and BR algorithms are close (98.129%, 0.0622, 97.849%, 0.0587, 98.151%, and 0.0585, respectively), the LM training algorithm is the best in terms of speed of calculation and display of results.

Advanced PSO Algorithms Development with Combined lbest and gbest Neighborhood Topologies

Abstract This paper introduces an innovative approach integrating global best (gbest) and local best (lbest) PSO communication topologies. The algorithm initiates with lbest and seamlessly transitions to gbest, with the switching rate controlled by the parameter “a”. Rational values of “a” is determined through numerical experiments. A comparative methodology employing two estimation criteria is used to showcase the improved performance of the modified PSO-based algorithms. Furthermore, the efficacy of this approach is demonstrated in addressing two optimal control problems within dynamical systems. Results highlight the modified algorithms’ superiority in terms of the total number of successful runs and statistical indicators. Consequently, these advanced algorithms prove effective for applications such as artificial neural network training, controller gains determination, and similar problem domains.

Enhancing heart disease diagnosis through ECG image vectorization-based classification

Heart disease is a major issue, and the severity of its effects can be reduced through early detection and prevention. ECG is an effective diagnostic tool. Automating ECG analysis increases the possibility of timely analysis and prediction of heart conditions, improving patient outcomes. The extraction of heart-related features further enhances the accuracy of ECG-based classification models, paving the way for more effective and efficient online detection and prevention of heart diseases.The image-vectorization technique suggested in this study produces a vector representation that precisely captures the distinctive features of the heart signal. It involves image cropping, erasing the ECG grid lines, and assigning pixels to distinguish the heart signal from the background. Compared to the feature vector produced by VGG16, the extracted feature vector is 589 times shorter than the feature vector produced by VGG16, which significantly decreased the amount of memory required, increased algorithm convergence, and required less computing power. The feature vector extracted using image-vectorization is used to create the training dataset, which is used to train artificial neural networks (ANNs). The results demonstrate that using image-vectorization techniques improved the performance of machine learning algorithms compared to using conventional feature extraction algorithms like CNNs and VGG16.

Цифровое моделирование интерсубъективности в социальном конструировании и феноменологических анализах мира повседневности

The widespread introduction of digital information and communication tech­nologies is changing the very nature of interpersonal and mass social interaction, creating new practices, meanings and forms of intersubjectivity. The article ana­lyzes a number of approaches to the creation of general artificial intelligence (AGI) systems based on the convergence of methods of phenomenological ana­lysis of intentional structures of intersubjectivity with technologies for creating and training artificial neural network systems of generative intelligence – Chat­GPT. It is shown that one of the most promising areas of constructing the cogni­tive architecture of AGI is the theory of frames as modeling schemes or matrices of stereotypical organization and interpretation of everyday experience. This conclusion is based on the fact that frames, as well as the procedures for swit­ching and configuring them, act as a kind of “spiritual automata” that control typical behavior of people in typical situations. The dual role of digital platforms in the social construction of reality is shown. On the one hand, digital platforms allow users to realize their aspirations and express the uniqueness of their indi­viduality, facilitate access to information, etc. On the other hand, our digital alter Ego, generated by intelligent systems, acquires the ability to recursively target, direct and control our thinking, motivations, emotional reactions, etc. The direct digital access to our spiritual life is not an inevitable consequence of the intro­duction of digital technologies into the world of everyday life, but a political project that can and should be sought for effective alternatives

Cyclical Curriculum Learning.

Artificial neural networks (ANNs) are inspired by human learning. However, unlike human education, classical ANN does not use a curriculum. Curriculum learning (CL) refers to the process of ANN training in which samples are used in a meaningful order. When using CL, training begins with a subset of the dataset and new samples are added throughout the training, or training begins with the entire dataset and the number of samples used is reduced. With these changes in training dataset size, better results can be obtained with curriculum, anti-curriculum, or random-curriculum methods than the vanilla method. However, a generally efficient CL method for various architectures and datasets is not found. In this article, we propose cyclical CL (CCL), in which the data size used during training changes cyclically rather than simply increasing or decreasing. Instead of using only the vanilla method or only the curriculum method, using both methods cyclically like in CCL provides more successful results. We tested the method on 18 different datasets and 15 architectures in image and text classification tasks and obtained more successful results than no-CL and existing CL methods. We also have shown theoretically that it is less erroneous to apply CL and vanilla cyclically instead of using only CL or only the vanilla method. The code of the cyclical curriculum is available at https://github.com/CyclicalCurriculum/Cyclical-Curriculum.

Training Artificial Neural Networks to Detect Multiple Sclerosis Lesions Using Granulometric Data from Preprocessed Magnetic Resonance Images with Morphological Transformations

The symptoms of multiple sclerosis (MS) are determined by the location of demyelinating lesions in the white matter of the brain and spinal cord. Currently, magnetic resonance imaging (MRI) is the most common tool used for diagnosing MS, understanding the course of the disease, and analyzing the effects of treatments. However, undesirable components may appear during the generation of MRI scans, such as noise or intensity variations. Mathematical morphology (MM) is a powerful image analysis technique that helps to filter the image and extract relevant structures. Granulometry is an image measurement tool for measuring MM that determines the size distribution of objects in an image without explicitly segmenting each object. While several methods have been proposed for the automatic segmentation of MS lesions in MRI scans, in some cases, only simple data preprocessing, such as image resizing to standardize the input dimensions, has been performed before the algorithm training. Therefore, this paper proposes an MRI preprocessing algorithm capable of performing elementary morphological transformations in brain images of MS patients and healthy individuals in order to delete undesirable components and extract the relevant structures such as MS lesions. Also, the algorithm computes the granulometry in MRI scans to describe the size qualities of lesions. Using this algorithm, we trained two artificial neural networks (ANNs) to predict MS diagnoses. By computing the differences in granulometry measurements between an image with MS lesions and a reference image (without lesions), we determined the size characterization of the lesions. Then, the ANNs were evaluated with the validation set, and the performance results (test accuracy = 0.9753; cross-entropy loss = 0.0247) show that the proposed algorithm can support specialists in making decisions to diagnose MS and estimating the disease progress based on granulometry values.

Integrating 3DVAR and ANN for inversion optimization of physical models of fuel-assembly bowing

This study introduces a novel inversion optimization method for the core physical model of fuel-assembly bowing, integrating measurement data based on three-dimensional variational (3DVAR) data assimilation and artificial neural network (ANN) techniques. Recognizing the deviation between theoretical physical models and actual core conditions due to factors like uneven coolant distribution and fuel-assembly bowing, this work aims to enhance model reliability and simulation accuracy. By generating a dataset through random perturbation and employing ANN for training, the method establishes a relationship between the core physical model and model-simulated values corresponding to measurement data. The construction of the cost function, essential for optimizing the inversion process, is executed using 3DVAR. To verify its validity, the method is implemented, and an inversion optimization process is developed using our self-developed Bamboo-C code system. The optimization process, validated through application in a commercial pressurized water reactor (PWR) in a nuclear power plant (NPP) in China, demonstrates a significant reduction in the relative errors of power distribution, thus affirming the method’s efficacy in achieving a more realistic model of fuel-assembly bowing. The integration of measurement data into the optimization process addresses the critical need for accurate core physical models in nuclear engineering, offering a promising avenue for enhancing the safety and efficiency of nuclear reactor operations.