Neural Network Simulation Research Articles

Neural Networks to Solve Partial Differential Equations: A Comparison With Finite Elements

We compare the Finite Element Method (FEM) simulation of a standard Partial Differential Equation thermal problem of a plate with a hole with a Neural Network (NN) simulation. The largest deviation from the true solution obtained from FEM ($0.015$ for a solution on the order of unity) is easily achieved with NN too without much tuning of the hyperparameters. Accuracies below $0.01$ instead require refinement with an alternative optimizer to reach a similar performance with NN. A rough comparison between the Floating Point Operations values, as a machine-independent quantification of the computational performance, suggests a significant difference between FEM and NN in favour of the former. This also strongly holds for computation time: for an accuracy on the order of $10^{-5}$, FEM and NN require $54$ and $1100$ seconds, respectively. A detailed analysis of the effect of varying different hyperparameters shows that accuracy and computational time only weakly depend on the major part of them. Accuracies below $0.01$ cannot be achieved with the "adam'' optimizers and it looks as though accuracies below $10^{-5}$ cannot be achieved at all. In conclusion, the present work shows that for the concrete case of solving a steady-state 2D heat equation, the performance of a FEM algorithm is significantly better than the solution via networks.

Detection and Selection of Moving Objects in Video Images Based on Impulse and Recurrent Neural Networks

The purpose of the article is to develop a methodology for automating the detection and selection of moving objects. The detection and separation of moving objects based on impulse and recurrence neural networks simulation. The result of the work is a developed motion detector based on impulse and recurrence neural networks and an automated system developed on the basis of this detector for detecting and separating moving objects and is ready for practical application. The feasibility of integrating the developed motion detector with Emgu CV (OpenCV) image processing package, multimedia framework functions, and DirectShow application programming interface were investigated. The proposed approach and software for the detection and separating of moving objects in video images using neural networks can be integrated into more sophisticated specialized computer-aided video surveillance systems, IoT (Internet of Things), IoV (Internet of Vehicles), etc.

Simulation and experimental study of effect of vortex finder structural parameters on cyclone separator performance

This study attempts to clarify the correlation between the structural parameters of vortex finders (diameter ratio and insertion depth) and the strong turbulent flow field and performance of cyclone separators. To elucidate the effects due to the overflow pipe diameter, insertion depth, and Reynolds number on the tangential velocity index (n) and separation efficiency, 0.3D, 0.4D, 0.5D, 0.6D, and 0.7D (where D is the diameter of the cyclone separator’s cylindrical section) were selected as the vortex finder diameter and insertion depth, which were studied using three methods: computational fluid dynamics (CFD) software Fluent, experiments, and artificial neural network simulations. With Fluent, the Reynolds stress model (RSM) of turbulence was adopted to describe transport equations, and the discrete phase model (DPM) for two-phase flow was used to calculate the efficiency and separate particles at different inlet velocities. Experiments were conducted to measure the operational pressure drop and separation efficiency of the system; subsequently, the results were used to validate the CFD calculation results. Finally, a 5–5-1 three-layer artificial neural network was established based on 336 sets of data for values of n obtained from numerical simulations. In addition, a backpropagation (BP) artificial neural network model was employed to predict the correlation between the structural and operational parameters of the cyclone separator and n values. The obtained results indicate that the BP artificial neural network can accurately predict the distribution of n values within the flow field of the cyclone. Increasing the diameter of the vortex finder with the same inlet velocity can reduce the total pressure drop due to the cyclone, thereby reducing the kinetic energy loss and improving the separation efficiency of fine particles. Increasing the insertion depth of the vortex finder leads to an increase in the values of n. Considering the stability of the flow field structure and energy dissipation, an S/D value between 0.5 and 0.6 is preferred.

Modeling and simulation of all-optical diffractive neural network based on nonlinear optical materials.

In this Letter, we propose an all-optical diffractive deep neural network modeling method based on nonlinear optical materials. First, the nonlinear optical properties of graphene and zinc selenide (ZnSe) are analyzed. Then the optical limiting effect function corresponding to the saturation absorption coefficient of the nonlinear optical materials is fitted. The optical limiting effect function is taken as the nonlinear activation function of the neural network. Finally, the all-optical diffractive neural network model based on nonlinear materials is established. The numerical simulation results show that the model can effectively improve the nonlinear representation ability of the all-optical diffractive neural network. It provides a theoretical support for the further realization of a photonic artificial intelligence chip based on nonlinear optical materials.

Compact model of retention characteristics of ferroelectric FinFET synapse with MFIS gate stack

In this paper, multiple-fin n- and p-channel HfZrO2 ferroelectric-FinFET devices are manufactured using a gate first process with post metalization annealing. The device transfer characteristics upon program and erase operations are measured and modeled. The drift in the transfer characteristics due to depolarization field and charge injection are captured using the shift in the threshold voltage along with time-dependent modeling of vertical field dependent mobility degradation parameters to develop a physical, computationally efficient, and accurate retention model for ferroelectric-FinFET devices. The modeled conductance is incorporated into deep neural network simulation platform CIMulator to analyze the role of conductance drift due to retention degradation, as well as the importance of the gap between high and low conductance states in improving the image recognition accuracy of neural networks.

Heat transport in liquid water from first-principles and deep neural network simulations

We compute the thermal conductivity of water within linear response theory from equilibrium molecular dynamics simulations, by adopting two different approaches. In one, the potential energy surface (PES) is derived on the fly from the electronic ground state of density functional theory (DFT) and the corresponding analytical expression is used for the energy flux. In the other, the PES is represented by a deep neural network (DNN) trained on DFT data, whereby the PES has an explicit local decomposition and the energy flux takes a particularly simple expression. By virtue of a gauge invariance principle, established by Marcolongo, Umari, and Baroni, the two approaches should be equivalent if the PES were reproduced accurately by the DNN model. We test this hypothesis by calculating the thermal conductivity, at the GGA (PBE) level of theory, using the direct formulation and its DNN proxy, finding that both approaches yield the same conductivity, in excess of the experimental value by approximately 60%. Besides being numerically much more efficient than its direct DFT counterpart, the DNN scheme has the advantage of being easily applicable to more sophisticated DFT approximations, such as meta-GGA and hybrid functionals, for which it would be hard to derive analytically the expression of the energy flux. We find in this way, that a DNN model, trained on meta-GGA (SCAN) data, reduce the deviation from experiment of the predicted thermal conductivity by about 50%, leaving the question open as to whether the residual error is due to deficiencies of the functional, to a neglect of nuclear quantum effects in the atomic dynamics, or, likely, to a combination of the two.

Application of neural network modeling and constrained particle swarm optimization of steam production from biomass power plant

In this paper, the steam production of the power plant from four biomass types has been investigated via the combination of the neural network (NN) model and constrained particle swarm optimization (CPSO) methods. First, the power plant modeling was accurately developed using NN simulation. Then, the model was applied to the optimization step, where the CPSO (the modified algorithm from the classical PSO) was introduced. The proposed method included real operation constraints to the algorithm, such as biomass feed availability and minimum steam production rate. In the optimization step, the COP has been defined as the cost function to check the present operation’s economic viability. To obtain the minimum production cost (COP), the CPSO suggested feeding wood bark from sources C and D with the amount of 104.1 and 463.8 tons/day, respectively. The result has been extensively validated by the real operation. The calculation framework in this study then applies to various production systems. Besides, by integrating the cost function in the optimization model with other indicators such as environmental impact indices, the output would provide an additional holistic picture of economic and environmental aspects.

Ultrasonic and mechanical properties of binary zinc-borate glasses using artificial neural networks simulation

The simulation-based artificial neural networks (ANN) program is one of the suitable candidates from artificial intelligence simulation which can work to predict important ultrasonic and mechanical parameters in the glass field. This research is focused on exploring the validity of this system by comparing the prediction values from ANN with the experimental measurements and other theoretical models. The ANN simulation was effectively applied to a binary zinc-borate glass system with the composition of zZnO−(100-z)B2O3 where z = 0, 40, 45, 50, 55, and 60 mol%, which was fabricated by using the melt-quenching techniques. The increase of ZnO content caused the ultrasonic velocity and elastic moduli of the glasses to exhibit a decreasing trend. The bond compression theoretical calculation compared with the experimental measurement was considered to be satisfactory with the value of the coefficient R2 being around 0.92452 to 0.98492. The Makishima-Mackenzie calculation model concerning the experimental measurement of the elastic moduli and Poisson's ratio were between 0.86628 to 0.99786. The coefficient R2 value from the ANN simulation displayed on the density, ultrasonic velocity, and elastic moduli graph is around 0.9999 to 1.0000, which is considered to be very reasonable. The values predicted by this remarkable model proved that ANN simulation is suitable for use in glass research.

Hierarchical prediction of soil water content time series

Quantifying the spatial and temporal dynamics of soil moisture is an important subject in vadose zone hydrology. A 1282-day field study was conducted to provide a hierarchy of data to assess neural network simulations of field soil water content time series θ(t). Volumetric water content was determined, typically once a week, by neutron thermalization at 20-, 40-, 60-, 80- and 100-cm depths. Soil samples were taken at 60 locations between 14- and 114-cm depths to determine soil properties, water retention, and saturated hydraulic conductivity. Prediction of hydraulic parameters from basic and extended soil properties yielded low correlation coefficients. Water content could be predicted reasonably with neural networks from soil properties or hydraulic parameters (0.880 < R < 0.942). Prediction of θ(t) based solely on rainfall data was not accurate. Independent networks could accurately simulate water content from one observed θ(t) based on the Nash-Sutcliffe Model Efficiency and Percent Bias. Once a network is trained for a particular depth, accurate predictions can be made beyond the training period using observations at just one location.

Determination of essential elements and trace heavy metals in agricrops by photoneutron activation analysis and X-ray fluorescence methods

Applicative researches in the field of environmental protection like X-ray fluorescence analysis, neutron activation analysis, and other methods, present real increasing importance related to the climate changes that can be observed nowadays. Scientific methods will enter step by step in our life, first of all, due to the accelerated technological development and due to the objective motivations of environmental monitoring necessary to take correct measures for the preservation and protection of nature on the Earth. Determination of some inorganic components, mainly heavy metals, in agricultural crops, is frequently required in health-related environmental studies, due to the high toxicity of trace amounts of such elements for the human organism. The main sources of trace elements to agricrops are their growing media, as soil-water-air ecosystem from which those nutrients are taken up by the root to the foliage. The goal of the present research was to determine the extent to which industrial inorganic pollutants are transferred to the crops. It was achieved by using multielement techniques as photon neutron activation method and X-ray fluorescence methods and statistical modeling in order to determine levels, pathways, and fate of toxic and non-toxic bioactive elements in selected agricrops along with the root soil. Some of the experimental trace metal values exceeded the threshold established by Romanian and EU regulations to protect vegetation and explain the estimated significant crop losses. Multivariate modeling by factor analysis and neural network simulation of the elemental concentration data showed always the component loaded with specific elements coming from industrial emissions. These kinds of studies are very requested regarding the vegetable meant for the human diet.

Multifunctional Optoelectronic Synapse Based on Ferroelectric Van der Waals Heterostructure for Emulating the Entire Human Visual System

AbstractThe development of optoelectronic synapses can provide an important breakthrough toward creating a sophisticated and adaptable artificial visual system analogous to that of humans. However, it remains a great challenge to implement the various functions of the biological visual neuromorphic system at the single device level. Intriguingly, 2D van der Waals (vdW) heterostructure may offer a platform to address the issue. Here, a novel multifunctional synaptic device based on ferroelectric α‐In2Se3/GaSe vdW heterojunction is proposed to emulate the entire biological visual system. Essential synaptic behaviors are observed in response to light and electrical stimuli; additionally, the retina‐like selectivity for light wavelengths and the achievement of Pavlov's dog experiment demonstrate the device's capacity for processing complex electrical and optical inputs. Beyond the optoelectronic synaptic behaviors, the device incorporates memory and logic functions analogous to those in the brain's visual cortex. The results of artificial neural network simulations show that the vdW heterojunction‐based device is completely capable of performing logic operations and recognizing images with a high degree of accuracy. The study indicates that versatile devices with a rationally designed construction have great potential for efficiently processing complex visual information and may simplify the design of artificial visual systems.

The Impact of Device Uniformity on Functionality of Analog Passively-Integrated Memristive Circuits

Passively-integrated memristors are the most prospective candidates for designing high-speed, energy-efficient, and compact neuromorphic circuits. Despite all the promising properties, experimental demonstrations of passive memristive crossbars have been limited to circuits with few thousands of devices until now, which stems from the strict uniformity requirements on the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IV</i> characteristics of memristors. This paper expands upon this vital challenge and investigates how uniformity impacts the computing accuracy of analog memristive circuits, focusing on neuromorphic applications. Specifically, the paper explores the tradeoffs between computing accuracy, crossbar size, switching threshold variations, and target precision. All-embracing simulations of matrix multipliers and deep neural networks on CIFAR-10 and ImageNet datasets have been carried out to evaluate the role of uniformity on the accuracy of computing systems. Further, we study three post-fabrication methods that increase the accuracy of nonuniform 0T1R neuromorphic circuits: hardware-aware training, improved tuning algorithm, and switching threshold modification. The application of these techniques allows us to implement advanced deep neural networks with almost no accuracy drop, using current state-of-the-art analog 0T1R technology.

Modeling Analysis and Comparision of Neural Network Simulation Based on ECM and LSTM

Comparing the prediction effects of traditional econometric algorithm model and deep learning algorithm model, taking regional GDP as an example, two prediction models of ARMA-ECM and LSTM-SVR are established for prediction, and the prediction results of different models are compared and analyzed. The results show that there are some deviations in the prediction results of the two models, but the prediction trends are the same. The prediction accuracy of LSTM-SVR model will decrease significantly with the reduction of time series data samples, while ARMA-ECM model is not so sensitive.

Hardware error correction for programmable photonics

Programmable photonic circuits of reconfigurable interferometers can be used to implement arbitrary operations on optical modes, providing a flexible platform for accelerating tasks in quantum simulation, signal processing, and artificial intelligence. A major obstacle to scaling up these systems is static fabrication error, where small component errors within each device accrue to produce significant errors within the circuit computation. Mitigating this error usually requires numerical optimization dependent on real-time feedback from the circuit, which can greatly limit the scalability of the hardware. Here we present a deterministic approach to correcting circuit errors by locally correcting hardware errors within individual optical gates. We apply our approach to simulations of large scale optical neural networks and infinite impulse response filters implemented in programmable photonics, finding that they remain resilient to component error well beyond modern day process tolerances. Our results highlight a potential way to scale up programmable photonics to hundreds of modes with current fabrication processes.

Adaptive Prediction of Enhanced Oil Recovery by N2 huff-n-puff in Fractured-Cavity Reservoir Using an FNN-FDS Hybrid Model

N2 huff-n-puff has proven to be a promising technique to further improve oil recovery in naturally fractured-cavity carbonate reservoirs. The effect of enhanced oil recovery (EOR) by N2 huff-n-puff is significantly affected by various dynamic and static factors such as type of reservoir space, reservoir connectivity, water influx, operational parameters, and so on, typically leading to a significant increase in oil production. To reduce the prediction uncertainty of EOR performance by N2 huff-n-puff, an adaptive hybrid model was proposed based on the fundamental principles of fuzzy neural network (FNN) and fractional differential simulation (FDS); a detailed prediction process of the hybrid model was also illustrated. The accuracy of the proposed FNN-FDS hybrid model was validated using production history of N2 huff-n-puff in a typical fractured-cavity carbonate reservoir. The proposed model was also employed to predict the EOR performance by N2 huff-n-puff in a naturally fractured-cavity carbonate reservoir. The methodology can serve as an effective tool to optimize developmental design schemes when using N2 huff-n-puff to tap more remaining oil in similar types of carbonate reservoirs.

Ultrafast machine vision with artificial neural network devices based on a GaN-based micro-LED array.

GaN-based micro-LED is an emerging display and communication device, which can work as well as a photodetector, enabling possible applications in machine vision. In this work, we measured the characteristics of micro-LED based photodetector experimentally and proposed a feasible simulation of a novel artificial neural network (ANN) device for the first time based on a micro-LED based photodetector array, providing ultrafast imaging (∼133 million bins per second) and a high image recognition rate. The array itself constitutes a neural network, in which the synaptic weights are tunable by the bias voltage. It has the potentials to be integrated with novel machine vision and reconfigurable computing applications, acting as a role of acceleration and similar functionality expansion. Also, the multi-functionality of micro-LED broadens its application potentials of combining ANN with display and communication.

Predictive modeling and simulation of wastewater treatment process using nano-based materials: Effect of pH and adsorbent dosage

A machine learning simulation via artificial neural network was implemented in this work to analyse adsorption removal of cadmium ions from aqueous solutions. The considered solid adsorbent in this work was a nanocomposite of mesoporous silica type and a polymer. To separate ions from water the mesoporous nanocomposite was selected due to its large pores. The simulation runs were carried out to correlate the adsorption parameters including solution pH and adsorbent dosage to the equilibrium concentration of ions in the liquid phase. The training and validation of the neural network indicated agreement with the experimental data for adsorption of cadmium ions, with high accuracy. The statistical evaluation of the data indicated great accuracy with R2 greater than 0.999 for the fitting. The findings of the modelling indicated that the effect of adsorbent dosage is not significant compared to the solution’s pH. The adsorption removal was increased with enhancing the solution pH due to the interaction between the ions and the surface of adsorbent. The results revealed that the developed neural network simulation is robust for prediction of ions removal from water using nanocomposite adsorbents.

EDHA: Event-Driven High Accurate Simulator for Spike Neural Networks

In recent years, spiking neural networks (SNNs) have attracted increasingly more researchers to study by virtue of its bio-interpretability and low-power computing. The SNN simulator is an essential tool to accomplish image classification, recognition, speech recognition, and other tasks using SNN. However, most of the existing simulators for spike neural networks are clock-driven, which has two main problems. First, the calculation result is affected by time slice, which obviously shows that when the calculation accuracy is low, the calculation speed is fast, but when the calculation accuracy is high, the calculation speed is unacceptable. The other is the failure of lateral inhibition, which severely affects SNN learning. In order to solve these problems, an event-driven high accurate simulator named EDHA (Event-Driven High Accuracy) for spike neural networks is proposed in this paper. EDHA takes full advantage of the event-driven characteristics of SNN and only calculates when a spike is generated, which is independent of the time slice. Compared with previous SNN simulators, EDHA is completely event-driven, which reduces a large amount of calculations and achieves higher computational accuracy. The calculation speed of EDHA in the MNIST classification task is more than 10 times faster than that of mainstream clock-driven simulators. By optimizing the spike encoding method, the former can even achieve more than 100 times faster than the latter. Due to the cross-platform characteristics of Java, EDHA can run on x86, amd64, ARM, and other platforms that support Java.

Robust neuromorphic coupled oscillators for adaptive pacemakers

Neural coupled oscillators are a useful building block in numerous models and applications. They were analyzed extensively in theoretical studies and more recently in biologically realistic simulations of spiking neural networks. The advent of mixed-signal analog/digital neuromorphic electronic circuits provides new means for implementing neural coupled oscillators on compact, low-power, spiking neural network hardware platforms. However, their implementation on this noisy, low-precision and inhomogeneous computing substrate raises new challenges with regards to stability and controllability. In this work, we present a robust, spiking neural network model of neural coupled oscillators and validate it with an implementation on a mixed-signal neuromorphic processor. We demonstrate its robustness showing how to reliably control and modulate the oscillator’s frequency and phase shift, despite the variability of the silicon synapse and neuron properties. We show how this ultra-low power neural processing system can be used to build an adaptive cardiac pacemaker modulating the heart rate with respect to the respiration phases and compare it with surface ECG and respiratory signal recordings from dogs at rest. The implementation of our model in neuromorphic electronic hardware shows its robustness on a highly variable substrate and extends the toolbox for applications requiring rhythmic outputs such as pacemakers.

Early Detection of Diabetic Retinopathy

Diabetic retinopathy are among the most common causes of vision loss in today's world. Visual impairment impacts about one in 3 diabetics, according to an epidemiological research. Diagnostic imaging is an important aspect of medical photography in contemporary world. Deep learning improves the eyesight for identifying illness in radiography. The goal is to use machine learning to diagnose vision loss. Deep learning in diagnostic devices might improve and speed up the diagnosis of sugar-related vision loss. This research will look at neural network models, algorithms, and simulations in order to diagnose diabetic retinopathy rapidly and help the medical system. The classifier is constructed using CNN.