Abstract Artificial Neural Networks Research Articles

One Memristor-One Electrolyte-Gated Transistor based High Energy-Efficient Dropout Neuronal Units

Abstract Artificial neural networks (ANN) have been extensively researched due to their significant energy-saving benefits. Hardware implementations of ANN with dropout function would be able to avoid the overfitting problem. This letter reports a dropout neuronal unit (1R1T-DNU) based on one memristor-one electrolyte-gated transistor with an ultralow energy consumption of 25 pJ/spike. A dropout neural network is constructed based on such device and has been verified by MNIST dataset, demonstrating high recognition accuracies (>90%) within a large range of dropout probabilities up to 40%. The running time can be reduced by increasing dropout probability without a significant loss in accuracy. Our results indicate the great potential of introducing such 1R1T-DNUs in full-hardware neural networks to enhance energy efficiency and to solve the overfitting problem.

123 Benchmarking of Artificial Neural Network Models for Genomic Prediction of Quantitative Traits in Pigs

Abstract Artificial neural networks (ANN) are a type of machine learning model that has been applied to various genomic problems, with the ability to learn non-linear relationships and model high-dimensional data. ANNs also have the potential in genomic prediction by capturing the intricate relationship between genetic variants and phenotypes. However, there is currently a limited effort to investigate the performance and feasibility of ANNs for pig genomic predictions. In this study, we evaluated the predictive performance of TensorFlow’s ANN models with one-layer, two-layer, and three-layer structures (with zero, one, and two hidden layers, respectively), in comparison with five linear methods, including GBLUP, LDAK, BayesR, SLEMM and scikit-learn’s ridge regression using data of six quantitative traits including off-test body weight (WT), off-test back fat thickness (BF), off-test loin muscle depth (MS), number of piglets born alive (NBA), number of piglets born dead (NBD), and number of piglets weaned (NW). Furthermore, we assessed the computational efficiency of ANNs on both CPU and GPU. The benchmarking was based on cross-validations of 26,190 genotyped pigs. We employed hyperband tuning to optimize the hyper-parameters and select the best model among one-layer, two-layer, and three-layer structures. Results showed that the one-layer structure, which is equivalent to ridge regression, yielded the best performance comparable to that of GBLUP. Using the optimal hyper-parameters for two-layer and three-layer structures, ANNs underperformed GBLUP in terms of accuracy. Of the five linear methods, BayesR and SLEMM performed similarly and the best, followed by LDAK, scikit-learn’s ridge regression, and GBLUP. Moreover, SLEMM was the fastest, which completed training with 21k individuals and 30k SNPs in 2.6 minutes. Compared with CPUs, GPUs exhibited a comparable computational speed for one-layer ANN but offered significant gains in computational efficiency for multi-layer ANNs. Based on our analysis of optimal hyper-parameters for two-layer ANN with BF, we found that using a GPU can lead to a five-fold increase in processing speed compared with using a conventional CPU, but it is still slower than GBLUP. In addition, hyper-parameter tuning (particularly for L2 regularization and the number of dense units in hidden layers) is critical for improving the genomic prediction accuracy in pigs. In conclusion, we found ANN with up to three layers could not improve genomic predictions compared with routine linear methods for pig quantitative traits.

Improving the performance of artificial neural networks trained on synthetic data in gas spectroscopy – a study on two sensing approaches

Abstract Artificial neural networks (ANNs) are used in quantitative infrared gas spectroscopy to predict concentrations on multi-component absorption spectra. Training of ANNs requires vast amounts of labelled training data which may be elaborate and time consuming to obtain. Additional data can be gained by the utilization of synthetically generated spectra, but at the cost of systematic deviations to measured data. Here, we present two approaches to train ANNs with a combination of comparatively small, measured data sets and synthetically generated data. For the first approach a neural network is trained hybridly with synthetically generated infrared absorption spectra of mixtures of N2O and CO and measured zero-gas spectra, taken with a mid-infrared dual comb spectrometer. This improves the mean absolute error (MAE) of the network predictions from 0.46 to 0.01 ppmV and 0.24 to 0.01 ppmV for the concentration predictions of N2O and CO respectively for zero-gas measurements which was previously observed for training with purely synthetic data. At the same time a similar performance on spectra from gas mixtures of 0–100 ppmV N2O and 0 to 60 ppmV CO was achieved. For the second approach an ANN pre-trained on synthetic infrared spectra of mixtures of acetone and ethanol is retrained on a small dataset consisting of 26 spectra taken with a mid-infrared photoacoustic spectrometer. In this case the MAE for the concentration predictions of ethanol and acetone are improved by 45 % and 20 % in comparison to purely synthetic training. This shows the capability of using synthetically generated data to train ANNs in combination with small amounts of measured data to further improve neural networks for gas sensing and the transferability between different sensing approaches.

Economic Nowcasting with Long Short-Term Memory Artificial Neural Networks (LSTM)

Abstract Artificial neural networks (ANNs) have been the catalyst to numerous advances in a variety of fields and disciplines in recent years. Their impact on economics, however, has been comparatively muted. One type of ANN, the long short-term memory network (LSTM), is particularly well-suited to deal with economic time-series. Here, the architecture’s performance and characteristics are evaluated in comparison with the dynamic factor model (DFM), currently a popular choice in the field of economic nowcasting. LSTMs are found to produce superior results to DFMs in the nowcasting of three separate variables; global merchandise export values and volumes, and global services exports. Further advantages include their ability to handle large numbers of input features in a variety of time frequencies. A disadvantage is the stochastic nature of outputs, common to all ANNs. In order to facilitate continued applied research of the methodology by avoiding the need for any knowledge of deep-learning libraries, an accompanying Python (Hopp 2021a) library was developed using PyTorch. The library is also available in R, MATLAB, and Julia.

Optimization of ANN Architecture and Training Parameters Using Taguchi Method

Artificial neural networks (ANNs) have gained significant popularity for modeling and optimization. Choosing appropriate training and design parameters for an ANN, on the other hand, is still a difficult task. These parameters are typically chosen by a trial-and-error strategy in which a large number of ANN models are developed and compared. This evidence stated how the Taguchi approach can be used to enhance an ANN model. To show the technique, a case study of a minimization of Green Sand-casting defects in a green sand-casting process is provided. Total 175 samples were taken for the study. L18 orthogonal array was used to construct the ANN training and architectural parameters. A Taguchi-optimized ANN model was created and showed excellent prediction accuracy. Analyses and experiments have shown that the best ANN training and architectural parameters may be found in a systematic manner, minimizing the time-consuming trial-and-error method.

River flow modelling for flood prediction using artificial neural network in ungauged Perkerra catchment, Baringo County, Kenya

Abstract Artificial neural network (ANN) modelling has been applied successfully in hydrology to predict future flows based on previous rainfall-runoff values. For a long time, flooding has been experienced in the surrounding areas of the Rift Valley lakes including Lake Baringo, fed by the River Perkerra, due to the rising water levels because of the above-normal rainfall season, resulting in massive socioeconomic losses. The study aims at predicting the occurrence of floods in River Perkerra using an ANN model with the input data being 417 consistent pairs of daily rainfall and discharge, and simulated runoff as the output. The model was trained, tested and validated producing a best fit regression with R2 of 0.951 for training, 0.938 for validation, 0.953 for testing giving an average of 0.949 indicating a close relationship between the input and output values. The overall best validation performance, RMSE, was 0.9204 m3/s indicating high efficiency of the FFNN model developed to predict floods. Flows greater than 14 m3/s, Q1, were the extreme flood events closely associated with socioeconomic losses. This prediction of Q1 value is crucial in the formulation and implementation of measures and policies by the County Government that will mitigate adverse impacts of predicted floods in the catchment.

Evaluation Model of Student Satisfaction in International Student Education Based on Neural Networks

Artificial neural network (ANN) theory is a rapidly developing information science around the world. The Back Propagation (BP) network is one of many artificial neural network types. It is a multilayer feedforward network with sophisticated nonlinear mapping capabilities. It is actually a relatively complex and nonlinear comprehensive decision-making problem when taking into account the comprehensive evaluation of test scores based on evaluation indicators that reflect the quality of the test papers. The five major factors of education service quality are tentatively proposed in this paper, which uses a neural network to model and analyze it. This paper examines the five factors of the education service quality evaluation model one by one and analyzes the issues that exist in the education service quality of international students in China. According to empirical research, overall satisfaction with the quality of education service in Guangxi among foreign students in China is low, and subindicators of education service quality are unbalanced. This paper analyzes the reasons for this situation from the three levels of service consciousness, emotional investment, and professional quality and proposes that the main task of improving education service quality is to optimize the entire service process, in order to narrow the gap between foreign students’ expectations and actual perceptions of educational services.

Multimodal transistors as ReLU activation functions in physical neural network classifiers

Artificial neural networks (ANNs) providing sophisticated, power-efficient classification are finding their way into thin-film electronics. Thin-film technologies require robust, layout-efficient devices with facile manufacturability. Here, we show how the multimodal transistor’s (MMT’s) transfer characteristic, with linear dependence in saturation, replicates the rectified linear unit (ReLU) activation function of convolutional ANNs (CNNs). Using MATLAB, we evaluate CNN performance using systematically distorted ReLU functions, then substitute measured and simulated MMT transfer characteristics as proxies for ReLU. High classification accuracy is maintained, despite large variations in geometrical and electrical parameters, as CNNs use the same activation functions for training and classification.

Artificial neural network tools for predicting the functional response of ultrafast laser textured/structured surfaces

Artificial Neural Networks (ANNs) are well-established knowledge acquisition systems with proven capacity for learning and generalisation. Therefore, ANNs are widely applied to solve engineering problems and are often used in laser-based manufacturing applications. There are different pattern recognition and control problems where ANNs can be effectively applied, and one of them is laser structuring/texturing for surface functionalisation, e.g. in generating Laser-Induced Periodic Surface Structures (LIPSS). They are a particular type of sub-micron structures that are very sensitive to changes in laser processing conditions due to processing disturbances like varying Focal Offset Distance (FOD) and/or Beam Incident Angle (BIA) during the laser processing of 3D surfaces. As a result, the functional response of LIPSS-treated surfaces might be affected, too, and typically needs to be analysed with time-consuming experimental tests. Also, there is a lack of sufficient process monitoring and quality control tools available for LIPSS-treated surfaces that could identify processing patterns and interdependences. These tools are needed to determine whether the LIPSS generation process is in control and consequently whether the surface’s functional performance is still retained. In this research, an ANN-based approach is proposed for predicting the functional response of ultrafast laser structured/textured surfaces. It was demonstrated that the processing disturbances affecting the LIPSS treatments can be classified, and then, the surface response, namely wettability, of processed surfaces can be predicted with a very high accuracy using the developed ANN tools for pre- and post-processing of LIPSS topography data, i.e. their areal surface roughness parameters. A Generative Adversarial Network (GAN) was applied as a pre-processing tool to significantly reduce the number of required experimental data. The number of areal surface roughness parameters needed to fully characterise the functional response of a surface was minimised using a combination of feature selection methods. Based on statistical analysis and evolutionary optimisation, these methods narrowed down the initial set of 21 elements to a group of 10 and 6 elements, according to redundancy and relevance criteria, respectively. The validation of ANN tools, using the salient surface parameters, yielded accuracy close to 85% when applied for identification of processing disturbances, while the wettability was predicted within an r.m.s. error of 11 degrees, equivalent to the static water contact angle (CA) measurement uncertainty.

Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots

AbstractArtificial neural networks (ANNs) can be employed as controllers for robotic agents. Their structure is often complex, with many neurons and connections, especially when the robots have many sensors and actuators distributed across their bodies and/or when high expressive power is desirable. Pruning (removing neurons or connections) reduces the complexity of the ANN, thus increasing its energy efficiency, and has been reported to improve the generalization capability, in some cases. In addition, it is well-known that pruning in biological neural networks plays a fundamental role in the development of brains and their ability to learn. In this study, we consider the evolutionary optimization of neural controllers for the case study of Voxel-based soft robots, a kind of modular, bio-inspired soft robots, applying pruning during fitness evaluation. For a locomotion task, and for centralized as well as distributed controllers, we experimentally characterize the effect of different forms of pruning on after-pruning effectiveness, life-long effectiveness, adaptability to new terrains, and behavior. We find that incorporating some forms of pruning in neuroevolution leads to almost equally effective controllers as those evolved without pruning, with the benefit of higher robustness to pruning. We also observe occasional improvements in generalization ability.

ARTIFICIAL NEURAL NETWORK (ANN) AND ITS APPLICATIONS IN THE FIELD OF PHARMACY WITH A SPECIAL FOCUS ON PHARMACEUTICAL PRODUCT DEVELOPMENT: A RECENT REVIEW

Artificial Neural Network (ANN) is a new field and a machine learning method based on artificial intelligence (AI) which has recently been utilized to optimize pharmaceutical products. ANN usually makes highly complicated models into a simple and convenient numerical solution. The potential of ANN technique is its capacity to understand and measure complicated non-linear input output interactions and their ability to interpret ambiguous or slightly constricted structures. Description about the ANN includes the historical background, its different phases while product development and also its various types that can be employed in field of optimized pharmaceutical product development. Within the context of Quality by Design (QbD), ANN techniques can be employed for the creation of design space within the limits of all applied variable factors. The aim of this review paper includes the description about the ANN and all possible applications in the field of pharmaceutical product development.

Chaos-generalized regression neural network prediction model of mine water inflow

Artificial neural network (ANN) provides a new way for mine water inflow prediction. However, the effectiveness of prediction using ANN model would not be guaranteed if the influencing factors of water inflow are difficult to quantify or there are only a few observation data. Chaos theory can recover the rich dynamic information hidden in time series. By reconstructing water inflow time series in phase space, the multi-dimensional matrix could be obtained, with each column representing an influencing factor of water inflow and its value representing the change of the influencing factor with time. Therefore, a new prediction model of mine water inflow can be established by combining ANN with chaos theory when lacking data on the influencing factors of water inflow. In the present study, the No. 12 coal mine of Pingdingshan China was selected as the study site. The Chaos-GRNN model and Chaos- BPNN model of mine, water inflow were established by using the water inflow data from February 1976 to December 2013. The model was verified by using the water inflow values in the 24 months from 2014 to 2015. The number embedded dimension (M) of influencing factors of water inflow determined by phase space reconstruction was 7, meaning that there were 7 influencing factors of water inflow and 7 neurons in GRNN input layer, and the time delay was 13 months. The value of GRNN input layer neurons was determined accordingly. The maximum Lyapunov index was 0.0530, and the prediction time of GRNN was 19 months. The two models were evaluated by using four evaluation indices (R, RMSE, MAPE, NSE) and violin plot. It was found that both models can realize the long-term prediction of water inflow, and the prediction effectiveness of Chaos-GRNN model is better than that of Chaos-BPNN model.

Combining genetic risk score with artificial neural network to predict the efficacy of folic acid therapy to hyperhomocysteinemia

Artificial neural network (ANN) is the main tool to dig data and was inspired by the human brain and nervous system. Several studies clarified its application in medicine. However, none has applied ANN to predict the efficacy of folic acid treatment to Hyperhomocysteinemia (HHcy). The efficacy has been proved to associate with both genetic and environmental factors while previous studies just focused on the latter one. The explained variance genetic risk score (EV-GRS) had better power and could represent the effect of genetic architectures. Our aim was to add EV-GRS into environmental factors to establish ANN to predict the efficacy of folic acid therapy to HHcy. We performed the prospective cohort research enrolling 638 HHcy patients. The multilayer perception algorithm was applied to construct ANN. To evaluate the effect of ANN, we also established logistic regression (LR) model to compare with ANN. According to our results, EV-GRS was statistically associated with the efficacy no matter analyzed as a continuous variable (OR = 3.301, 95%CI 1.954–5.576, P < 0.001) or category variable (OR = 3.870, 95%CI 2.092–7.159, P < 0.001). In our ANN model, the accuracy was 84.78%, the Youden’s index was 0.7073 and the AUC was 0.938. These indexes above indicated higher power. When compared with LR, the AUC, accuracy, and Youden’s index of the ANN model (84.78%, 0.938, 0.7073) were all slightly higher than the LR model (83.33% 0.910, 0.6687). Therefore, clinical application of the ANN model may be able to better predict the folic acid efficacy to HHcy than the traditional LR model. When testing two models in the validation set, we got the same conclusion. This study appears to be the first one to establish the ANN model which added EV-GRS into environmental factors to predict the efficacy of folic acid to HHcy. This model would be able to offer clinicians a new method to make decisions and individual therapeutic plans.

Research on improved convolutional wavelet neural network

Artificial neural networks (ANN) which include deep learning neural networks (DNN) have problems such as the local minimal problem of Back propagation neural network (BPNN), the unstable problem of Radial basis function neural network (RBFNN) and the limited maximum precision problem of Convolutional neural network (CNN). Performance (training speed, precision, etc.) of BPNN, RBFNN and CNN are expected to be improved. Main works are as follows: Firstly, based on existing BPNN and RBFNN, Wavelet neural network (WNN) is implemented in order to get better performance for further improving CNN. WNN adopts the network structure of BPNN in order to get faster training speed. WNN adopts the wavelet function as an activation function, whose form is similar to the radial basis function of RBFNN, in order to solve the local minimum problem. Secondly, WNN-based Convolutional wavelet neural network (CWNN) method is proposed, in which the fully connected layers (FCL) of CNN is replaced by WNN. Thirdly, comparative simulations based on MNIST and CIFAR-10 datasets among the discussed methods of BPNN, RBFNN, CNN and CWNN are implemented and analyzed. Fourthly, the wavelet-based Convolutional Neural Network (WCNN) is proposed, where the wavelet transformation is adopted as the activation function in Convolutional Pool Neural Network (CPNN) of CNN. Fifthly, simulations based on CWNN are implemented and analyzed on the MNIST dataset. Effects are as follows: Firstly, WNN can solve the problems of BPNN and RBFNN and have better performance. Secondly, the proposed CWNN can reduce the mean square error and the error rate of CNN, which means CWNN has better maximum precision than CNN. Thirdly, the proposed WCNN can reduce the mean square error and the error rate of CWNN, which means WCNN has better maximum precision than CWNN.

New neural network classification method for individuals ancestry prediction from SNPs data

Artificial Neural Network (ANN) algorithms have been widely used to analyse genomic data. Single Nucleotide Polymorphisms(SNPs) represent the genetic variations, the most common in the human genome, it has been shown that they are involved in many genetic diseases, and can be used to predict their development. Developing ANN to handle this type of data can be considered as a great success in the medical world. However, the high dimensionality of genomic data and the availability of a limited number of samples can make the learning task very complicated. In this work, we propose a New Neural Network classification method based on input perturbation. The idea is first to use SVD to reduce the dimensionality of the input data and to train a classification network, which prediction errors are then reduced by perturbing the SVD projection matrix. The proposed method has been evaluated on data from individuals with different ancestral origins, the experimental results have shown the effectiveness of the proposed method. Achieving up to 96.23% of classification accuracy, this approach surpasses previous Deep learning approaches evaluated on the same dataset.

Genetic Algorithmised Neuro Fuzzy System for Forecasting the Online Journal Visitors

Artificial Neural Network (ANN) is recognized as one of effective forecasting engines for various business fields. This approach fits well with non-linear data. In fact, it is a black box system with random weighting, which is hard to train. One way to improve its performance is by hybridizing ANN with other methods. In this paper, a hybrid approach, Genetic Algorithm-Neural Fuzzy System (GA-NFS) is proposed to predict the number of unique visitors of an online journal website. The neural network weight is precisely determined using GA. Afterwards, the best weight has been used for testing data and processed using Sugeno Fuzzy Inference System (FIS) for time-series forecasting. Based on experiment, GA-NFS have been produced accuracy with 0.989 of root mean square error (RMSE) that is lower than the RMSE of a common NFS (2,004). This may indicate that the GA based weighting is able to improve the NFS performance on forecasting the number of journal unique visitors.

Prediction of the Work-Related Injuries Based on Neural Networks

Abstract Artificial neural networks (ANN) are a powerful tool in the decision-making process, especially in solving the complex problems with a large number of input data. The possibility to predict the work-related injuries in the underground coal mines, based on application of the neural networks, is analyzed in this work. the input data for the network were obtained based on a survey of 1300 respondents. After analyzing the input data influence on the network output, 14 most influential inputs were selected, with help of which the network correctly predicted whether the worker would suffer the work-related injury or not, with 80% precision. The two models were developed, based on the multilayer perceptron (MLP) and radial basis function (RBF) networks. The two models’ results were compared to each other. The sensitivity analysis was used to select the most influential parameters, like mine, age of miners, as well as their work experience. The parameters were further analyzed by use of the descriptive statistics. The selected parameters are direct indicators of problems that can cause injuries. The obtained results point to the fact that the work-related injuries can be successfully predicted by application of the artificial neural networks. The proposed models’ importance is reflected in the clear indicators for enforcing the stricter occupational safety and organizational measures in order to reduce the number of work-related injuries in underground mines.

Artificial neural network classification of asteroids in the M1:2 mean-motion resonance with Mars

ABSTRACT Artificial neural networks (ANNs) have been successfully used in the last years to identify patterns in astronomical images. The use of ANN in the field of asteroid dynamics has been, however, so far somewhat limited. In this work, we used for the first time ANN for the purpose of automatically identifying the behaviour of asteroid orbits affected by the M1:2 mean-motion resonance with Mars. Our model was able to perform well above 85 per cent levels for identifying images of asteroid resonant arguments in term of standard metrics like accuracy, precision, and recall, allowing to identify the orbital type of all numbered asteroids in the region. Using supervised machine learning methods, optimized through the use of genetic algorithms, we also predicted the orbital status of all multi-opposition asteroids in the area. We confirm that the M1:2 resonance mainly affects the orbits of the Massalia, Nysa, and Vesta asteroid families.

LONG-TERM PREDICTION OF CASPIAN SEA LEVEL UNDER CMIP6 SCENARIOS USING ARTIFICIAL NEURAL NETWORKS

Artificial Neural Network (ANN) is employed to predict the long-term Caspian Sea level (CSL). 114-year observed CSL data (1900-2014) and the precipitation and temperature of historical and future scenarios of Coupled Model Intercomparison Phase 6 (CMIP6) are used to predict the future fluctuations of CSL (2015-2050). The values of the statistical indices in training, validating and testing periods (1900-2014) indicate the efficiency of the ANN in reconstruction of the CSL. Considering the outputs of different climate change projections (CMIP6) and excluding the human interventions, the study predicts the CSL fluctuation range of -28 m to -26m until 2050.Recorded Presentation from the vICCE (YouTube Link): https://youtu.be/Kfj-gr65TR8

Two Machine Learning Based Schemes for Solving Direct and Inverse Problems of Radiative Transfer Theory

Artificial neural networks (ANNs) are used to substitute computationally expensive radiative transfer models (RTMs) and inverse operators (IO) for retrieving optical parameters of the medium. However, the direct parametrization of RTMs and IOs by means of ANNs has certain drawbacks, such as loss of generality, computations of huge training datasets, robustness issues etc. This paper provides an analysis of different ANN-related methods, based on our results and those published by other authors. In particular, two techniques are proposed. In the first method, the ANN substitutes the eigenvalue solver in the discrete ordinate RTM, thereby reducing the computational time. Unlike classical RTM parametrization schemes based on ANN, in this method the resulting ANN can be used for arbitrary geometry and layer optical thicknesses. In the second method, the IO is trained by using the real measurements (preprocessed Level-2 TROPOMI data) to improve the stability of the inverse operator. This method provides robust results even without applying the Tikhonov regularization method.