Implementation Of Artificial Neural Networks Research Articles

Li-Ion Batteries Parameter Estimation With Tiny Neural Networks Embedded on Intelligent IoT Microcontrollers

Lithium-ion (Li-Ion) batteries are rechargeable batteries which can maximize battery lifespan thanks to their chemical abilities, at the same time increasing power energy density. For these reasons, Li-Ion batteries have earned considerable popularity, and they are widely used both in mobile computing devices (e.g. smartphones and smartwatches) and automotive systems (e.g. hybrid and electric vehicles). A fundamental parameter for battery health monitoring is the State of Health (SoH), which is computed from the maximum releasable capacity, and which represents battery functionality in energy storage and delivery. Among the most used data-driven approaches are Machine Learning (ML) algorithms, such as Support Vector Machines (SVMs), Random Forest (RF) regressions, and Artificial Neural Networks (ANNs). This article presents a comparison of different ML algorithms for estimating maximum releasable capacity of Li-Ion batteries, with a special focus on the implementation of both Forward and Recurrent ANNs (FNNs and RNNs, respectively), using prognostic Li-Ion battery data sets provided by the National Aeronautics and Space Administration (NASA). After an evaluation of models performances in terms of RMSE and MAE, STM32Cube.AI tool was used to convert pre-trained ANNs to optimized ANSI C code for STM32 microcontrollers (MCUs), and to profile their complexity automatically. Finally, in order to decrease models size with minimal accuracy loss, the implemented ANNs were quantized via STM32Cube.AI, converting weights and activations from 32-bit floating-point to 8-bit integer precision. TensorFlow Lite for Microcontrollers (TFLM) was used as benchmark in the analysis and validation of both non-quantized and quantized models, and the performances obtained via STM32Cube.AI and TFLM were compared.

Multi-well placement optimisation using sequential artificial neural networks and multi-level grid system

This study suggests a sequential artificial neural network (ANN) method coupled with a multi-level grid system to optimise multi-well placement in petroleum reservoirs. As the number of scenarios for placing wells increases exponentially with the number of wells, the difficulty in finding the global optimum increases accordingly due to the intrinsic uncertainty of ANNs. The multi-level grid system can reduce the size of the search space by allocating only one well grid block per several grid blocks in the basic grid system. A higher level of grid system consists of finer grid blocks to gradually improve the resolution of the grid system. Repetitive implementation of the sequential ANN at each level of the grid system narrows the search space, and the global optimum is determined. The proposed algorithm is validated with applications to two- and three-infill-well problems in a coal-bed methane (CBM) reservoir. [Received: March 16, 2018; Accepted: September 19, 2018]

Prediction of electrodeposited copper ions in high speed jet electrodeposition for MEMS fabrication using ANN

High speed jet electrodeposition offers fabrication of Micro Electro-Mechanical System (MEMS) features. In this technique, high precision and accuracy can be acquired using ultrasonic and controlled environment. This research is an attempt to predict the deposition of copper ion on metallic aluminium substrate. For this, a data set has been generated by performing the experiment multiple times at different variable inputs condition. Consequently, Artificial Neural Network (ANN) has been deployed using MATLAB for prediction. ANN is used to train, test and validate the collected experimental data for development of a function which estimate the electrode deposition rate. The variables include the gap between copper electrode and substrate, voltage applied between electrodes and concentration of CuSO4·5H2O However, no prior data is available to compare the performance because no attempt of ANN implementation has been done specifically in High Speed Jet Electro Deposition (HSJED). The coefficient of correlation R for training, testing and validation proved to be satisfied and came out as 0.94162, 0.98755 and 0.98242 respectively. The performance parameter was Mean Square Error (MSE). The prediction model would help the researcher to know the copper deposition rate and save time and efforts required.

Analog IGZO Memristor With Extended Capabilities

In this paper an IGZO memristor with multiple states and analog tuning extended capability is reported. The device has a planar structure and is fabricated by magnetron sputtering on glass substrate. The device resistance could be gradually increased and decreased within the range of one order of magnitude. Larger resistance changes are also possible but they are mostly irreversible. The obtained memristor looks promising to be used as electronic synapse in hardware implemented artificial neural networks or for applications in analog computing and cryptography.

Progresses and outlook in neuromorphic devices

In this information era with bursting data, there are three fundamental problems limiting the further development of computing power: The gradually slowing down of Moore’s law, the rapidly increasing energy consumption while scaling down, and the restriction of data transfer between separated memory and processor known as “Von Neumann bottleneck”. To deal with massive data and make all things in our daily life interconnected, it is imperative to develop new generation of computing paradigms, for example, neuromorphic computing. Inspired by the human brain, neuromorphic computing has drawn extensive attention in recent years due to its high parallelism, low power consumption and in-memory computation, especially when the tide of artificial intelligence is sweeping across the globe now. Novel neuromorphic devices are key to the construction of neuromorphic computing systems, providing an efficient implementation of artificial neural network on chip. Besides, by simulating the behavior of biological neurons and synapses at the physical level, neuromorphic devices can enable a brand new computing method, which is thought to be an essential and promising way to build a brain-like system on chip. This paper focuses on the current research progresses and future research trends of neuromorphic devices. It summarizes the physical mechanisms of neuromorphic devices, according to which they can be divided into ion-migration device, material phase change device, electron-migration device, magnetic device and so on. It further details the inner dynamics happened in a single device by characterizing the changing process in transmission electron microscope. This paper also explains how these neuromorphic devices can be used for computing, where memristor, as the missing fourth basic circuit component, is used as a theoretical support. Through making up a crossbar structure, these devices can directly calculate vector-matrix multiplication by Ohm’s law and Kirchhoff’s law. It is an elegant way that can accelerate artificial neural network in parallel by computing in memory. In addition, exploiting the intrinsic dynamics in the devices can realize complex and interesting functions of biological neural networks, such as long-term plasticity, short-term plasticity and spike timing dependent plasticity. For example, a synaptic transistor based on two-dimensional materials WSe2 can simulate the biologically transport process of calcium ion, leading precisely regulated coexistence of long-term and short-term memory. Remarkably, a significant advance is realizing heterosynaptic plasticity, a general mode in brain’s cortical network, in a single multi-terminal device, which is crucial for biologically plausible supervised learning on hardware. Furthermore, artificial neurons composed of memristors and capacitors have some natural advantages compared to traditional circuits, containing rich dynamics for neuromorphic computing, such as stochasticity, adaptive threshold and chaotic oscillation. Recently, materials with new mechanisms, like ferroelectric materials, are also being explored to realize complex neuron functions, contributing to less power and smaller area. By devices and algorithms co-design, it is time to explore the next generation of neural network, transforming current processing unit to a more efficient and intelligent brain-inspired style. As a conclusion, the outstanding challenges and trends in the field of neuromorphic devices are discussed in this paper. These researches are promising for building a neuromorphic computer in the future, which will be complementary to classical computer and outperform in many tasks.

Developing an Intelligent System for Prediction of Optimal Dose of Warfarin in Iranian Adult Patients with Artificial Heart Valve

Introduction: Artificial intelligence (AI) research within medicine is growing rapidly. AI is poised to transform medical practice. AI has been studied in several areas of healthcare and medical practice, including diagnosing, treating and caring of patients. Warfarin is one of the most commonly prescribed oral anticoagulant. Among all anticoagulants, warfarin has long been listed among the top ten drugs causing adverse drug events. Due to narrow therapeutic range and significant side effects, warfarin dosage determination becomes a challenging task in clinical practice. The purpose of this study was to determine exact dose of warfarin needed for patients with artificial heart valve using artificial neural networks (ANN).Development: To achieved the best model, some multi-layer perceptron ANNs were constructed with different structures. The dataset used included 846 patients who had been referred to the PT clinic in Tehran heart center in the second six months of the year 2013. Finally, the best structure of ANN for warfarin dose was investigated and used for prediction system developments. In this paper the implementation of ANNs and proposed system in MatLab environment are described.Application: The effectiveness of ANNs were evaluated in terms of classification performance using 10fold cross-validation procedure and the results showed that the best model is a network that has 7 neurons in its hidden layer with an average absolute error of 0.1, disturbance rate of 0.33 and regression of 0.87. Conclusion: The achieved results reveal that ANN-based system is a suitable tool for warfarin dose prediction in Iranian patients with an artificial heartvalve. However, no system can be guaranteed to achieve 100% accuracy, but using such methods can reduce medical errors and thereby improve health care and patient safety.

Recognition of Activities of Daily Living and Environments Using Acoustic Sensors Embedded on Mobile Devices

The identification of Activities of Daily Living (ADL) is intrinsic with the user’s environment recognition. This detection can be executed through standard sensors present in every-day mobile devices. On the one hand, the main proposal is to recognize users’ environment and standing activities. On the other hand, these features are included in a framework for the ADL and environment identification. Therefore, this paper is divided into two parts—firstly, acoustic sensors are used for the collection of data towards the recognition of the environment and, secondly, the information of the environment recognized is fused with the information gathered by motion and magnetic sensors. The environment and ADL recognition are performed by pattern recognition techniques that aim for the development of a system, including data collection, processing, fusion and classification procedures. These classification techniques include distinctive types of Artificial Neural Networks (ANN), analyzing various implementations of ANN and choosing the most suitable for further inclusion in the following different stages of the developed system. The results present 85.89% accuracy using Deep Neural Networks (DNN) with normalized data for the ADL recognition and 86.50% accuracy using Feedforward Neural Networks (FNN) with non-normalized data for environment recognition. Furthermore, the tests conducted present 100% accuracy for standing activities recognition using DNN with normalized data, which is the most suited for the intended purpose.

Prediction of BLEVE mechanical energy by implementation of artificial neural network

In the event of a BLEVE, the overpressure wave can cause important effects over a certain area. Several thermodynamic assumptions have been proposed as the basis for developing methodologies to predict both the mechanical energy associated to such a wave and the peak overpressure. According to a recent comparative analysis, methods based on real gas behavior and adiabatic irreversible expansion assumptions can give a good estimation of this energy. In this communication, the Artificial Neural Network (ANN) approach has been implemented to predict the BLEVE mechanical energy for the case of propane and butane. Temperature and vessel filling degree at failure have been considered as input parameters (plus vessel volume), and the BLEVE blast energy has been estimated as output data by the ANN model. A Bayesian Regularization algorithm was chosen as the three-layer backpropagation training algorithm. Based on the neurons optimization process, the number of neurons at the hidden layer was five in the case of propane and four in the case of butane. The transfer function applied in this layer was a sigmoid, because it had an easy and straightforward differentiation for using in the backpropagation algorithm. For the output layer, the number of neurons had to be one in both cases, and the transfer function was purelin (linear). The model performance has been compared with experimental values, proving that the mechanical energy of a BLEVE explosion can be adequately predicted with the Artificial Neural Network approach.

Optimization of wireless power transfer using artificial neural network: A review

AbstractWireless power transfer (WPT) is widely explored and applied nowadays because of its simplicity in transferring power without using wire, easy maintenance, and equipment mobility. Due to mobility and compatibility attributes, WPT is utilized in powering biomedical devices, small electronic equipment, wireless sensor, mobile phones, and high voltage applications (eg, electric vehicles). The implementation of artificial neural network (ANN) in WPT has emerged as a powerful/prominent tool for estimating the performance parameters due to its learning and significant features. Such implementation can minimize design complexity and time‐consuming calculations. An early application of ANN employs the information derived from the collectively measured processes for training the ANN algorithm. After a suitable training process, the network output can be considered in place of computationally thorough representations to speed up the result search. To obtain precise result and optimize the parameters in WPT, several popular ANN algorithms have been used by researchers. This review paper highlighted the latest research specifically regarding the implementation of ANN in WPT, which included the types of ANN implemented in WPT, current WPT problem investigation that used ANN, and a comparison between the techniques. Moreover, the challenges and constraints of ANN techniques were elucidated at the end of this paper.

Energy-efficient Design of MTJ-based Neural Networks with Stochastic Computing

Hardware implementations of Artificial Neural Networks (ANNs) using conventional binary arithmetic units are computationally expensive, energy-intensive, and have large area overheads. Stochastic Computing (SC) is an emerging paradigm that replaces these conventional units with simple logic circuits and is particularly suitable for fault-tolerant applications. We propose an energy-efficient use of Magnetic Tunnel Junctions (MTJs), a spintronic device that exhibits probabilistic switching behavior, as Stochastic Number Generators (SNGs), which forms the basis of our NN implementation in the SC domain. Further, the error resilience of target applications of NNs allows approximating the synaptic weights in our MTJ-based NN implementation, in ways brought about by properties of the MTJ-SNG, to achieve energy-efficiency. An algorithm is designed that, given an error tolerance, can perform such approximations in a single-layer NN in an optimal way owing to the convexity of the problem formulation. We then use this algorithm and develop a heuristic approach for approximating multi-layer NNs. Classification problems were evaluated on the optimized NNs and results showed substantial savings in energy for little loss in accuracy.

Molecular and DNA Artificial Neural Networks via Fractional Coding.

This article considers implementation of artificial neural networks (ANNs) using molecular computing and DNA based on fractional coding. Prior work had addressed molecular two-layer ANNs with binary inputs and arbitrary weights. In prior work using fractional coding, a simple molecular perceptron that computes sigmoid of scaled weighted sum of the inputs was presented where the inputs and the weights lie between [-1,1]. Even for computing the perceptron, the prior approach suffers from two major limitations. First, it cannot compute the sigmoid of the weighted sum, but only the sigmoid of the scaled weighted sum. Second, many machine learning applications require the coefficients to be arbitrarily positive and negative numbers that are not bounded between [-1,1]; such numbers cannot be handled by the prior perceptron using fractional coding. This paper makes four contributions. First molecular perceptrons that can handle arbitrary weights and can compute sigmoid of the weighted sums are presented. Thus, these molecular perceptrons are ideal for regression applications and multi-layer ANNs. A new molecular divider is introduced and is used to compute sigmoid(ax) where . Second, based on fractional coding, a molecular artificial neural network (ANN) with one hidden layer is presented. Third, a trained ANN classifier with one hidden layer from seizure prediction application from electroencephalogram is mapped to molecular reactions and DNA and their performances are presented. Fourth, molecular activation functions for rectified linear unit (ReLU) and softmax are also presented.

Manipulation of the electrical behaviors of Cu/MXene/SiO2/W memristor

Electronic synapses with both long-term and short-term plasticity are considered as significant components for constructing brain-inspired computing systems. Research progress on electrical synapses have proved that memristors possess huge similarities with biological synapses. Nevertheless, an effective mean of manipulating the biological properties of memristors is still unclear. In this letter, we propose a memristor and reveal that the compliance current of electroforming plays an active role in tuning short-term and long-term plasticity of the memristors. The results may provide a useful guideline for manipulating memristor as electronic synapses in the hardware implementation of artificial neural networks.

Solar-stimulated optoelectronic synapse based on organic heterojunction with linearly potentiated synaptic weight for neuromorphic computing

We report an artificial optoelectronic synapse based on a copper-phthalocyanine (CuPc) and para-sexiphenyl (p-6P) heterojunction structure. This device features stable conductance states and their linear distribution in long-term potentiation (LTP) characteristic curve formed by continuous input light pulses. These superior synaptic characteristics originate from the fact that the number of photo-holes moving into the CuPc channel and photo-electrons being trapped at the p-6P/dielectric interface is constant at every light pulse. A single-layer neural network is theoretically formed with these optoelectronic synaptic devices and its feasibility is studied in terms of training/recognition tasks of the Modified National Institute of Standards and Technology digit image patterns. Owing to the excellent LTP characteristic and through the use of a unidirectional update method, its maximum recognition rate is as high as 78% despite the use of a single-layer network. This study is expected to provide a foundation for future studies on optoelectronic synaptic devices toward the implementation of complex artificial neural networks.

Toward Better PV Panel’s Output Power Prediction; a Module Based on Nonlinear Autoregressive Neural Network with Exogenous Inputs

Much work has been carried out for modeling the output power of photovoltaic panels. Using artificial neural networks (ANNS), one could efficiently model the output power of heterogeneous photovoltaic (HPV) panels. However, due to the existing different types of artificial neural network implementations, it has become hard to choose the best approach to use for a specific application. This raises the need for studies that develop models using the different neural networks types and compare the efficiency of these different types for that specific application. In this work, two neural network types, namely, the nonlinear autoregressive network with exogenous inputs (NARX) and the deep feed-forward (DFF) neural network, have been developed and compared for modeling the maximum output power of HPV panels. Both neural networks have four exogenous inputs and two outputs. Matlab/Simulink is used in evaluating the proposed two models under a variety of atmospheric conditions. A comprehensive evaluation, including a Diebold-Mariano (DM) test, is applied to verify the ability of the proposed networks. Moreover, the work further investigates the two developed neural networks using their actual implementation on a low-cost microcontroller. Both neural networks have performed very well; however, the NARX model performance is much better compared with DFF. Using the NARX network, a prediction of PV output power could be obtained, with half the execution time required to obtain the same prediction with the DFF neural network, and with accuracy of ±0.18 W.

A Historical Perspective on Hardware AI Inference, Charge-Based Computational Circuits and an 8 bit Charge-Based Multiply-Add Core in 16 nm FinFET CMOS

The second wave of AI is about statistical learning of low dimensional structures from high dimensional data. Inference is done using multilayer, data transforming networks using fixed point arithmetic with parameters that have limited precision (4–16 bits). In this paper we give a historical perspective on hardware AI inference deep Artificial Neural Networks (ANNs) or in short Deep Neural Networks (DNN) and deep learning. We review custom chip implementations of ANNs from thirty years ago and from the more recent publications in the last five years. With only few exceptions, hardware AI architectures are digital but we argue that if done right, i.e. in the charge domain, analog computation will have a role in future hardware AI systems. We make our discussion concrete by presenting the architecture, implementation and measurements from a mixed-signal, charge-based 8-bit analog multiplier for limited precision linear algebra in AI systems. Using a capacitor array, and charge redistribution, the architecture performs the multiplication operation in the charge domain at the thermal noise limit with near minimum energy dissipation. The charge redistribution multiplier core was fabricated in a 16 nm FinFET CMOS process, with measured energy 1.4 fJ for the analog multiplication operation. Compared to a conventional digital implementation synthesized and simulated in the same technology, the proposed design achieves the same performance at 37% less energy.

The Use of Artificial Neural Networks and Decision Trees to Predict the Degree of Odor Nuisance of Post-Digestion Sludge in the Sewage Treatment Plant Process

This paper presents the application of artificial neural networks and decision trees for the prediction of odor properties of post-fermentation sludge from a biological-mechanical wastewater treatment plant. The input parameters were concentrations of popular compounds present in the sludge, such as toluene, p-xylene, and p-cresol, and process parameters including the concentration of volatile fatty acids, pH, and alkalinity in the fermentation sludge. The analyses revealed that the implementation of artificial neural networks allowed the prediction of the values of odor intensity and the hedonic tone of the post-fermentation sludge at the level of 30% mean absolute percentage error. Application of the decision tree made it possible to determine what input parameters the fermentation feed should have in order to arrive at the post-fermentation sludge with an odor intensity <2 and hedonic tone >−1. It was shown that the aforementioned phenomenon was influenced by the following factors: concentration of p-xylene, pH, concentration of volatile fatty acids, and concentration of p-cresol.

Binarized Neural Network with Silicon Nanosheet Synaptic Transistors for Supervised Pattern Classification

In the biological neural network, the learning process is achieved through massively parallel synaptic connections between neurons that can be adjusted in an analog manner. Recent developments in emerging synaptic devices and their networks can emulate the functionality of a biological neural network, which will be the fundamental building block for a neuromorphic computing architecture. However, on-chip implementation of a large-scale artificial neural network is still very challenging due to unreliable analog weight modulation in current synaptic device technology. Here, we demonstrate a binarized neural network (BNN) based on a gate-all-around silicon nanosheet synaptic transistor, where reliable digital-type weight modulation can contribute to improve the sustainability of the entire network. BNN is applied to three proof-of-concept examples: (1) handwritten digit classification (MNIST dataset), (2) face image classification (Yale dataset), and (3) experimental 3 × 3 binary pattern classifications using an integrated synaptic transistor network (total 9 × 9 × 2 162 cells) through a supervised online training procedure. The results consolidate the feasibility of binarized neural networks and pave the way toward building a reliable and large-scale artificial neural network by using more advanced conventional digital device technologies.

Practical Implementation of Memristor-Based Threshold Logic Gates

Current advances in emerging memory technologies enable novel and unconventional computing architectures for high-performance and low-power electronic systems, capable of carrying out massively parallel operations at the edge. One emerging technology, ReRAM, also known to belong in the family of memristors (memory resistors), is gathering attention due to its attractive features for logic and in-memory computing and benefits which follow from its technological attributes, such as nanoscale dimensions, low-power operation, and multi-state programming. At the same time, the design with CMOS is quickly reaching its physical and functional limitations, and further research toward novel logic families, such as threshold logic gates (TLGs), is scoped. In this paper, we introduce a physical implementation of a memristor-based current-mode TLG (MCMTLG) circuit and validate its design and operation through multiple experimental setups. We demonstrate two-input, three-input, and four-input MCMTLG configurations and showcase their reconfiguration capability. This is achieved by varying memristive weights arbitrarily for shaping the classification decision boundary, thus showing a promise as an alternative hardware-friendly implementation of artificial neural networks. Through the employment of real memristor devices as the equivalent of synaptic weights in TLGs, we are realizing components that can be used toward an in silico classifier.

Implementation of Artificial Neural Network using Scaled Conjugate Gradient in ISO 9001:2015 Audit Findings Classification

Auditors of Quality Management System (QMS) face challenges in generating accurate audit reports due to some factors that can be attributed to technical competence, experience, time, auditee reaction, and other factors. Incorrect clauses cited in audit reports may result to loss of integrity of the auditor and the auditing procedure itself, hence, it is important that auditors should be careful in citing clauses of the standard to avoid chaos and complaints from auditees. To resolve this issue, this paper presents the implementation of Artificial Neural Network (ANN) using Scaled Conjugate Gradient (SCG) algorithm to classify audit findings based on the clauses of the ISO 9001:2015 QMS Requirements international standard. In this paper, the author explored how the neural network can predict the correct clause of the standard according to text patterns of audit findings. Based on modelling results, the neural network has generated a Cross Entropy (CE) values of 6.39, 18.09, 18.09 and Percentage Error (PE) values of 21.83, 21.58, and 22.39 in training, testing, and validation environments, respectively. Moreover, the model has achieved a combined Classification Accuracy (CA) of 96%, as for which, based on the actual implementation, the model has accurately predicted 95% of the audit findings analyzed.

Ionic floating-gate memories

Device Technology Digital implementations of artificial neural networks perform many tasks, such as image recognition and language processing, but are too energy intensive for many applications. Analog circuits that use large crossbar arrays of synaptic memory elements represent a low-power alternative, but most devices cannot update the synaptic weights uniformly or scale to large array sizes. Fuller et al. developed an integrated device, ionic floating-gate memory, that has the gate terminal of a redox transistor electrically connected to a diffusive memristor. This low-power device enabled linear and symmetric weight updates in parallel over an entire crossbar array at megahertz rates over 109 write-read cycles. Science , this issue p. [570][1] [1]: /lookup/doi/10.1126/science.aaw5581