Artificial Neural Network Training Research Articles

Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN.

Spiking neural networks (SNNs) have shown advantages in computation and energy efficiency over traditional artificial neural networks (ANNs) thanks to their event-driven representations. SNNs also replace weight multiplications in ANNs with additions, which are more energy-efficient and less computationally intensive. However, it remains a challenge to train deep SNNs due to the discrete spiking function. A popular approach to circumvent this challenge is ANN-to-SNN conversion. However, due to the quantization error and accumulating error, it often requires lots of time steps (high inference latency) to achieve high performance, which negates SNN's advantages. To this end, this paper proposes Fast-SNN that achieves high performance with low latency. We demonstrate the equivalent mapping between temporal quantization in SNNs and spatial quantization in ANNs, based on which the minimization of the quantization error is transferred to quantized ANN training. With the minimization of the quantization error, we show that the sequential error is the primary cause of the accumulating error, which is addressed by introducing a signed IF neuron model and a layer-wise fine-tuning mechanism. Our method achieves state-of-the-art performance and low latency on various computer vision tasks, including image classification, object detection, and semantic segmentation. Codes are available at: https://github.com/yangfan-hu/Fast-SNN.

Artificial Neural Networks versus Multiple Linear Regressions to Predict the Christiansen Uniformity Coefficient in Sprinkler Irrigation

The Christiansen Uniformity Coefficient (CUC) describes the distribution of water in a sprinkler system. In this study, two types of models were developed to predict the Christiansen Uniformity Coefficient (CUC) of sprinkler irrigation systems: Artificial Neural Network (ANN), specifically the feed-forward neural networks, and multiple linear regression (MLR) models. The models were trained on a dataset of published research on the CUC of sprinkler irrigation systems, which included data on a variety of design, operating, and meteorological condition variables. In order to build the predictive model of CUC, 10 input parameters were used including sprinkler height (H), working pressure (P), nozzle diameter (D and da), sprinkler line spacing (SL), sprinkler spacing (SS), wind speed (WS), wind direction (WD), temperature (T), and relative humidity (RH). Fifty percent (50%) of the data was used to train ANN models and the remaining data for cross-validation (25%) and for testing (25%). Multiple linear regression models were built using the training data. Four statistical criteria were used to evaluate the model’s predictive quality: the correlation coefficient (R), the index of agreement (d), the root mean square error (RMSE), and the mean absolute error (MAE). Statistical analysis demonstrated that the best predictive ability was obtained when the models (ANN and MLR) utilized all the input variables. The results demonstrated that the accuracy of ANN models, predicting the CUC of sprinkler irrigation systems, is higher than that of the MLR ones. During the training stage, the ANN models were more accurate in predicting CUC than MLR, with higher R (0.999) and d (0.999) values and lower MAE (0.167) and RMSE (0.456) values. The R values of the MLR model fluctuated between 0.226 and 0.960, the d values oscillated from 0.174 to 0.979, the MAE values were in the range of 2.458% and 10.792%, and the RMSE values fluctuated from 2.923% to 13.393%. Furthermore, the study revealed that WS and WD are the most influential climatic parameters. The ANN model can be used to develop more accurate tools for predicting the CUC of sprinkler irrigation systems. This can help farmers to design and operate their irrigation systems more efficiently, which can save them time and money.

Machine learning models for early prediction of mortality risk in patients with burns: A single center experience

Mortality rate is considered as the most important outcome measure for assessing the severity of burn injury. A scale or model that accurately predicts burn mortality can be useful to determine the clinical course of burn injuries, discuss treatment options and rehabilitation with patients and their families, and evaluate novel, innovative interventions for the injuries. This study aimed to use machine learning models to predict the mortality risk of patients with burns after their first admission to the center and to compare the performances of these models. Overall, 1064 patients hospitalized in burn intensive care and burn service units between 2016 and 2022 were included in the study. In total, 40 parameters, including demographic characteristics and biochemical parameters of all patients, were analyzed in the study. Furthermore, the dataset was randomly divided into two clusters with 70% of the data used for artificial neural networks (ANNs) training and 30% for model success testing. The ANN model proposed in this study showed high success across all machine learning methods tried in different variants, with an accuracy of 95.92% in the test set. Machine learning models can be used to predict the mortality risk of patients with burns. This study may help validate the use of machine learning models for applications in clinical practice. Conducting multicenter studies will further contribute to the literature.

Data-driven inverse design of composite triangular lattice structures

In this work, we introduce a class of novel bi-material composite triangular lattice structures. The inverse design of these structures is achieved by using a data-driven method. They exhibit a broad range of tunable effective elastic properties, i.e., the effective Young’s and shear moduli span a few orders of magnitude, and the effective Poisson’s ratio can be both negative and positive. We exploit the computational homogenization method to calculate the effective elastic constants of these structures with varying structural features to generate a representative dataset. Subsequently, we harness the dataset to train artificial neural network models for both forward prediction and inverse design. The forward model predicts the effective properties of a given structure, while the inverse model generates a structure design for the specified target properties. We validate the performance of these models by several examples such as optimizing the isotropic auxetic properties. The data-driven surrogate models greatly facilitate the practical application of these novel lattice structures for various structural and/or functional purposes.

Estimation of the geometric measure of entanglement with Wehrl moments through artificial neural networks

In recent years, artificial neural networks (ANNs) have become an increasingly popular tool for studying problems in quantum theory, and in particular entanglement theory. In this work, we analyse to what extent ANNs can accurately predict the geometric measure of entanglement of symmetric multiqubit states using only a limited number of Wehrl moments (moments of the Husimi function of the state) as input, which represents partial information about the state. We consider both pure and mixed quantum states. We compare the results we obtain by training ANNs with the informed use of convergence acceleration methods. We find that even some of the most powerful convergence acceleration algorithms do not compete with ANNs when given the same input data, provided that enough data is available to train these ANNs. We also provide an experimental protocol for measuring Wehrl moments, which is state-independent. More generally, this work opens up perspectives for the estimation of entanglement measures and other SU(2)-invariant quantities, such as the Wehrl entropy, in a way that is more accessible in experiments than by means of full state tomography.

Dual‐Mode Operation of Epitaxial Hf0.5Zr0.5O2: Ferroelectric and Filamentary‐Type Resistive Switching

Since the discovery of its ferroelectricity, hafnium oxide is widely used for applications in ferroelectric field‐effect transistors and ferroelectric tunnel junctions. is especially favored for its robust ferroelectricity and high remanent polarization at low thicknesses. In addition, is well established as amorphous or crystalline oxide layer in resistive switching devices. Herein, ferroelectric switching is found coexisting with high on/off ratio resistive switching in sub‐10 nm epitaxial . The resistive switching shows typical characteristics of a filamentary‐type valence change memory (VCM), clearly contradicting the polarization charges as the origin of different resistance states. In contrast to previous observations, no electroforming step is required to initiate VCM switching. The bottom electrode enables a RESET to the virgin state, allowing subsequent ferroelectric hysteresis measurements. It is possible to change between both switching schemes repeatedly without impacting the ferroelectric performance. This indicates that ferroelectric switching and oxygen vacancy movement do not interfere with each other, and both switching phenomena can exist independently. This finding opens up ways to unite the different strengths of both switching mechanisms in the same stack. It becomes possible to assign the two operating principles to artificial neural network training and inference according to their respective advantages.

Smart Control Solution for Single-Stage Solar PV Systems

Solar photovoltaic (PV) systems unpredictable characteristics and tight grid-codes demand power electronic-based energy conversion devices. Hence, as the power levels generated by the solar PV systems rise, multi-level voltage source converters (VSC) and their control mechanisms become more necessary for effective energy conversion. Continuous control set model predictive control (CCS-MPC) is a class of predictive control approach that has emerged recently for the applications of power converters and energy conversion systems. In this paper, an artificial neural network (ANN) based controller for single-stage grid-connected PV is implemented. The CCS-MPC is used as an expert / a teacher to generate the data required for off-line training of the neural network controller. After the off-line training, the trained ANN can fully control the inverter’s output voltage and track the maximum power point (MPP) without the need for MPC during testing. The proposed control technique is assessed under various operating conditions. Based on the results obtained, it is observed that the proposed techniques offer improved objective tracking and comparative dynamic response with respect to the classical approaches.

Data driven development of iron-based metallic glasses using artificial neural networks

Metallic glasses exhibit outstanding properties due to their distinctive amorphous structure. Within the realm of metallic glasses, iron-based variants stand out as a cost-effective option. However, the conventional approach to discover metallic glass alloys is inefficient in regards to the required resources. Manufacturing processes for metallic glasses, such as melt spinning as well as the analysis by differential scanning calorimetry (DSC) are cost- and time-intensive. Artificial neural networks (ANN) are used to address this issue. The ANNs can predict the complex, non-linear behaviour of glass formation and therefore save time and resources. The database required for a Machine Learning approach was gathered through a literature research on iron-based metallic glasses. After hyperparameter tuning, the trained ANNs can estimate the Glass-Forming Ability (GFA) parameter γm of various alloy compositions with a Root Mean Squared Error of 0.0296. A Genetic Algorithm (GA) is incorporated in the ANN to find GFA maxima in defined ranges. This approach enables the optimisation of the GFA prior to experimentation and thus enhances the discovery of new iron-based metallic glass alloys. Using an ANN alleviates the need for in depth analysis and screening experimentation while still allowing a fast, resource efficient virtual alloy evaluation.

On the quality of training artificial neural networks without initial data preprocessing under conditions of their limited set

The task of training artificial neural networks (ANNs) for the diagnosis of Alzheimer's disease based on the magnetic susceptibility of 10 brain veins under conditions of limited initial data (81 patients) is considered. In a recent study, fully-connected ANNs with one and two hidden layers were successfully trained in a recent study after reducing the number of veins to 4 and determining the number of synaptic connections comparable to a neural network based on a decision root. The known results gave rise to the hypothesis that training ANNs on the same data without preprocessing would also yield high-quality neural networks. To assess the quality of ANNs, the following indicators were used: mean square errors of training and generalization (testing), coefficients of determination (R2) on the sets of values used in training and generalization (testing), as well as the number of trained ANNs satisfying a certain value of the above indicators. For graphical representation of the results of repeated experiments on training and testing of ANNs, a new graph is proposed to visualize the quality of ANNs. Thus, the paper demonstrates the quality indicators of 800 ANNs trained without data preprocessing, among which there were 7 neural networks with one hidden layer and 9 neural networks with two hidden layers, for each of which 50 experiments were conducted. For comparison, we present the quality indices of 2400 ANNs trained with preprocessing.

Strength and manufacturability enhancement of a composite automotive component via an integrated finite element/artificial neural network multi-objective optimization approach

This study addresses the enhancement of an injection-molded fiber-reinforced plastic / metal hybrid automotive structure and its plastic injection molding process through the integration of the finite element method, artificial intelligence and evolutionary search methods. Experiments are conducted to validate the finite element models. The orthogonal array and Latin hypercube methods are employed to generate a database via finite element analysis. The database is then used to train artificial neural networks that accurately evaluate component distortion, manufacturing time, and structural strength. A genetic optimization algorithm is applied to identify optimal process parameters. The procedure was demonstrated to simultaneously reduce product warpage and manufacturing time by 10 and 62 %, respectively, when compared with the reference manufacturing process while strength is kept above the required levels with a reduced number of required data points. A more in-depth investigation into the causes of strength variation and deformation is also provided. The results contribute to the advance of robust composite automotive structures with superior quality, manufactured through efficient processes.

Hybrid transformation with 1’st order statistics for medical image analysis and classification

Skin cancer, one of the most critical forms of cancer, required early detection and documentation for efficient treatment, especially as certain types are fatal. In this study, an artificial neural network (ANN) was utilized to discover and index diverse melanomas using the ISIC 2018 dataset. The pre-processing phase is stringent as it insulates the cancerous fraction of a skin image, involving removing, trimming, thinning, and normalizing. In this phase, unwanted hair pieces on the image are eliminated in this phase. Feature extraction from the clipped image is achieved using a discrete cosine transform (DCT) and a gradient transform to transform it into frequency domain coefficients. Statistical feature extraction is used to reduce the amount of data required for ANN training. A dataset from ISIC 2018 that consists of seven different images from dermoscopic procedures for classification purposes is used in the empirical investigation. An accuracy of 85.44% for DCT in the sub-bands and 76.07% for the sub-band gradient transform was achieved by the applied ANN. The hybrid system's mean squared error (MSE) was discovered to be 3.52×10-4. The work highlights the potential of ANN in the early detection of skin cancer, supporting more efficient treatment and preventing advanced cases.

Hybrid transformation with first order statistics for medical image analysis and classification

Skin cancer, one of the most critical forms of cancer, required early detection and documentation for efficient treatment, especially as certain types are fatal. In this study, an artificial neural network (ANN) was utilized to discover and index diverse melanomas using the ISIC 2018 dataset. The pre-processing phase is stringent as it insulates the cancerous fraction of a skin image, involving removing, trimming, thinning, and normalizing. In this phase, unwanted hair pieces on the image are eliminated in this phase. Feature extraction from the clipped image is achieved using a discrete cosine transform (DCT) and a gradient transform to transform it into frequency-domain coefficients. Statistical feature extraction is used to reduce the amount of data required for ANN training. A dataset from ISIC 2018 that consists of seven different images from dermoscopic procedures for classification purposes is used in the empirical investigation. An accuracy of 85.44% for DCT in the sub-bands and 76.07% for the sub-band gradient transform was achieved by the applied ANN. The hybrid system's mean squared error (MSE) was discovered to be 3.52×10-4. The work highlights the potential of ANN in the early detection of skin cancer, supporting more efficient treatment and preventing advanced cases.

New developments in the analysis of catch time series as the basis for fish stock assessments: The CMSY++ method

Following an introduction to the nature of fisheries catches and their information content, a new development of CMSY, a data-limited stock assessment method for fishes and invertebrates, is presented. This new version, CMSY++, overcomes several of the deficiencies of CMSY, which itself improved upon the “Catch-MSY” method published by S. Martell and R. Froese in 2013. The catch-only application of CMSY++ uses a Bayesian implementation of a modified Schaefer model, which also allows the fitting of abundance indices should such information be available. In the absence of historical catch time series and abundance indices, CMSY++ depends strongly on the provision of appropriate and informative priors for plausible ranges of initial and final stock depletion. An Artificial Neural Network (ANN) now assists in selecting objective priors for relative stock size based on patterns in 400 catch time series used for training. Regarding the cross-validation of the ANN predictions, of the 400 real stocks used in the training of ANN, 94% of final relative biomass (B/k) Bayesian (BSM) estimates were within the approximate 95% confidence limits of the respective CMSY++ estimate. Also, the equilibrium catch-biomass relations of the modified Schaefer model are compared with those of alternative surplus-production and age-structured models, suggesting that the latter two can be strongly biased towards underestimating the biomass required to sustain catches at low abundance. Numerous independent applications demonstrate how CMSY++ can incorporate, in addition to the required catch time series, both abundance data and a wide variety of ancillary information. We stress, however, the caveats and pitfalls of naively using the built-in prior options, which should instead be evaluated case-by-case and ideally be replaced by independent prior knowledge.

New developments in the analysis of catch time series as the basis for fish stock assessments: The CMSY++ method

Following an introduction to the nature of fisheries catches and their information content, a new development of CMSY, a data-limited stock assessment method for fishes and invertebrates, is presented. This new version, CMSY++, overcomes several of the deficiencies of CMSY, which itself improved upon the “Catch-MSY” method published by S. Martell and R. Froese in 2013. The catch-only application of CMSY++ uses a Bayesian implementation of a modified Schaefer model, which also allows the fitting of abundance indices should such information be available. In the absence of historical catch time series and abundance indices, CMSY++ depends strongly on the provision of appropriate and informative priors for plausible ranges of initial and final stock depletion. An Artificial Neural Network (ANN) now assists in selecting objective priors for relative stock size based on patterns in 400 catch time series used for training. Regarding the cross-validation of the ANN predictions, of the 400 real stocks used in the training of ANN, 94% of final relative biomass (B/k) Bayesian (BSM) estimates were within the approximate 95% confidence limits of the respective CMSY++ estimate. Also, the equilibrium catch-biomass relations of the modified Schaefer model are compared with those of alternative surplus-production and age-structured models, suggesting that the latter two can be strongly biased towards underestimating the biomass required to sustain catches at low abundance. Numerous independent applications demonstrate how CMSY++ can incorporate, in addition to the required catch time series, both abundance data and a wide variety of ancillary information. We stress, however, the caveats and pitfalls of naively using the built-in prior options, which should instead be evaluated case-by-case and ideally be replaced by independent prior knowledge.

Cooperative Computing Offloading Scheme via Artificial Neural Networks for Underwater Sensor Networks

Aiming at the problem of being unable to meet some high computing power, high-precision applications due to the limited capacity of underwater sensor nodes, and the difficulty of low computation power, in this paper, we introduce the edge servers, known as base stations for underwater sensor nodes, and propose a scheme to process the computational tasks based on coalition game theory. This scheme provides functions such as cooperation among different base stations within the coalition, the smart division of tasks, and efficient computational offloading. In order to reduce the complexity of the algorithm, the artificial neural network model is introduced into the method. Each task is divided into sub-parts and fed to an artificial neural network for training, testing, and validation. In addition, the scheme delivers the computed task from base stations back to sink nodes via a shortened path to enhance the service reliability. Due to the mobility of the base station in the ocean, our proposed scheme takes into account the dynamic environment at the same time. The simulation results show that, compared with the existing state-of-the-art methods, the success rate of our proposed approach improves by 30% compared with the Greedy method. The total service time of our proposed approach decreases by 12.6% compared with the Greedy method and 31.2% compared with the Always-Migrate method.

Effectiveness of neural networks and transfer learning to forecast photovoltaic power production

Artificial Neural Networks (ANNs) can successfully be integrated into smart models for energy prediction, but require large datasets for training. This investigation presents an innovative methodology for photovoltaic power generation forecasting with ANNs, when only a limited amount of real data is available, and has been tested and validated on a real-life photovoltaic installation. Feature selection identifies which meteorological features most impact photovoltaic power generation. A simulator, which accurately models a real photovoltaic installation, is used to create an artificial, but accurate and realistic, dataset of power generation large enough to effectively train and test different ANNs. These are then exploited on a portion of real, but limited, dataset of power generated by the real photovoltaic installation on which the simulator is modeled. Finally, different transfer learning techniques are used to tune the ANN models with the remaining portion of the real, but limited, dataset of photovoltaic power generation.

Diagnostic Signal Method for Fault Identification of Electro-Hydraulic Servo Actuators

In this paper, we develop a fault identification approach for electro-hydraulic servo actuators based on injecting a pre-defined diagnostic signal into the system and then extracting fault-related features from the phase space topology. Next, we build regression models using an artificial neural network, which maps the feature space to fault space to identify the faults represented by the system’s parameters. The performance of the proposed fault identification approach is evaluated when the degradation of permanent armature occurs. The effect of parametric faults on the dynamics is studied and discussed. The different excitation of the system is considered, and the robustness of the proposed method under the condition of noise is also explored. The obtained results indicate the effectiveness of injected diagnostic signals in enriching the dynamics of the system and increasing the quality of extracted features and the accuracy of trained artificial neural networks.

An Artificial Neural Network Model for the Stress Concentration Factors in KT-Joints Subjected to Axial Compressive Load

Stress concentration factor (SCF) is usually used to estimate the fatigue life of an offshore joint. Historically, parametric equations were used to estimate SCF based on a statistical analysis of experimental and finite element analysis (FEA) results, to reduce cost and time. These equations give the SCF at the saddle/crown position for simple joints and basic load cases. However, for modified or defective joints, the location of the maximum SCF can change. In such circumstances, the single-point SCF equation cannot be used to estimate the maximum value of SCF, as its location may have changed from saddle/crown. To our knowledge, there are no general expressions to estimate SCF around the brace axis accurately. As artificial neural networks (ANN) can approximate the trend of complex phenomena better than conventional data fitting, a mathematical model based on ANN is proposed to estimate SCF based on the weights and biases of trained ANN. Nine hundred thirty-seven finite element simulations were performed to generate SCF data for training the ANN. This ANN was used to model an empirical equation for SCF. The proposed empirical model can estimate SCF around the brace axis with less than 5% error. The current study provides a roadmap to using FEA and ANN for empirical modeling of SCF in tubular joints, and this approach can be applied to any joint type, with or without design modification or damage. Once a database of similar equations is available, it can be utilized for quickly estimating SCF instead of costly experimentation and FEA. Optimization of the ANN can further improve the accuracy of the developed mathematical model.

A unified approach of detecting phase transition in time-varying complex networks

Deciphering the non-trivial interactions and mechanisms driving the evolution of time-varying complex networks (TVCNs) plays a crucial role in designing optimal control strategies for such networks or enhancing their causal predictive capabilities. In this paper, we advance the science of TVCNs by providing a mathematical framework through which we can gauge how local changes within a complex weighted network affect its global properties. More precisely, we focus on unraveling unknown geometric properties of a network and determine its implications on detecting phase transitions within the dynamics of a TVCN. In this vein, we aim at elaborating a novel and unified approach that can be used to depict the relationship between local interactions in a complex network and its global kinetics. We propose a geometric-inspired framework to characterize the network’s state and detect a phase transition between different states, to infer the TVCN’s dynamics. A phase of a TVCN is determined by its Forman–Ricci curvature property. Numerical experiments show the usefulness of the proposed curvature formalism to detect the transition between phases within artificially generated networks. Furthermore, we demonstrate the effectiveness of the proposed framework in identifying the phase transition phenomena governing the training and learning processes of artificial neural networks. Moreover, we exploit this approach to investigate the phase transition phenomena in cellular re-programming by interpreting the dynamics of Hi-C matrices as TVCNs and observing singularity trends in the curvature network entropy. Finally, we demonstrate that this curvature formalism can detect a political change. Specifically, our framework can be applied to the US Senate data to detect a political change in the United States of America after the 1994 election, as discussed by political scientists.

Big Data and Machine Learning Framework for Temperature Forecasting

This research aims to develop a Supporting Big Data and ML with a Framework for temperature forecasting using Artificial Neural Networks (ANN). The proposed framework utilizes a massive amount of historical weather data to train the ANN model, which can effectively learn the complex non- correlations that are linear with the parameters and temperature. The input variables include various weather parameters, such as humidity, wind speed, precipitation, and pressure. The framework involves three main stages: data pre-processing, model training, and temperature forecasting. In the data pre-processing stage, the raw weather data is cleaned, normalized, and transformed into a suitable format for model training. The data is then split into training, validation, and testing sets to ensure model accuracy. In model instruction stage, the ANN trained model using a backpropagation algorithm to adjust affected by the inherent biases and model based on the input and output data. The training process is iterative, and Using the validation, the efficiency of the model is measured. set to prevent overfitting. Finally, in the temperature forecasting stage, the trained ANN model is used to predict the temperature for a given set of weather parameters. The accuracy of the temperature forecasting is evaluated using the testing set, and the results are compared to other forecasting methods, such as statistical methods and numerical weather prediction models. The proposed framework has several advantages over traditional temperature forecasting methods. Firstly, it utilizes a vast amount of data, which enhances the accuracy of the forecast. Secondly, the ANN model can learn the interactions between the input variables that are not linear and temperature, which cannot be captured by traditional statistical methods. Finally, the framework can be easily extended to incorporate additional weather parameters or to forecast other environmental variables. The results of this research show that the proposed framework can effectively forecast temperature with high accuracy, outperforming traditional statistical methods and numerical weather prediction models. Therefore, it has the potential to improve weather forecasting and contribute to various applications, such as agriculture, energy management, and transportation.