Realization Of Artificial Neural Networks Research Articles

Highly-dependable printed neuromorphic circuits based on additive manufacturing

The rapid development of emerging domains, such as the Internet of Things and wearable technologies, necessitates the development of flexible, stretchable, and non-toxic devices that can be manufactured at an ultra-low cost. Printed electronics has emerged as a viable solution by offering not only the aforementioned features but also a high degree of customization, which enables the personalization of products and facilitates the low-cost product development process even in small batches. In the context of printed electronics, printed neuromorphic circuits offer highly customized and bespoke realization of artificial neural networks to achieve desired functionality with very small number of hardware components. However, since analog components are utilized, the performance of printed neuromorphic circuits can be influenced by various factors. In this work, we focus on three main factors that perturb the circuit output from the designed values, namely, variations due to printing errors, aging effects of printed resistors, and input variations originating from sensing uncertainty. In the described approach, these variations are taken into account during the design (training) to ensure the dependability of the printed neuromorphic circuits. With this approach, the expected accuracy and the robustness of printed neural networks can be increased by 27% and 74%, respectively. Moreover, the ablation study suggests that, aging effect and printing variation may have similar effects on the functionality of printed neural networks. In contrast, the impact of sensing uncertainty on printed neural networks is almost orthogonal to aging and printing variations.

Connectome of memristive nanowire networks through graph theory

Hardware implementation of neural networks represents a milestone for exploiting the advantages of neuromorphic-type data processing and for making use of the inherent parallelism associated with such structures. In this context, memristive devices with their analogue functionalities are called to be promising building blocks for the hardware realization of artificial neural networks. As an alternative to conventional crossbar architectures where memristive devices are organized with a top-down approach in a grid-like fashion, neuromorphic-type data processing and computing capabilities have been explored in networks realized according to the principle of self-organization similarity found in biological neural networks. Here, we explore structural and functional connectivity of self-organized memristive nanowire (NW) networks within the theoretical framework of graph theory. While graph metrics reveal the link of the graph theoretical approach with geometrical considerations, results show that the interplay between network structure and its capacity to transmit information is related to a phase transition process consistent with percolation theory. Also the concept of memristive distance is introduced to investigate activation patterns and the dynamic evolution of the information flow across the network represented as a memristive graph. In agreement with experimental results, the emergent short-term dynamics reveals the formation of self-selected pathways with enhanced transport characteristics connecting stimulated areas and regulating the trafficking of the information flow. The network capability to process spatio-temporal input signals can be exploited for the implementation of unconventional computing paradigms in memristive graphs that take into advantage the inherent relationship between structure and functionality as in biological systems.

Memristive devices based on single ZnO nanowires—from material synthesis to neuromorphic functionalities

Memristive and resistive switching devices are considered promising building blocks for the realization of artificial neural networks and neuromorphic systems. Besides conventional top-down memristive devices based on thin films, resistive switching devices based on nanowires (NWs) have attracted great attention, not only for the possibility of going beyond current scaling limitations of the top-down approach, but also as model systems for the localization and investigation of the physical mechanism of switching. This work reports on the fabrication of memristive devices based on ZnO NWs, from NW synthesis to single NW-based memristive cell fabrication and characterization. The bottom-up synthesis of ZnO NWs was performed by low-pressure chemical vapor deposition according to a self-seeding vapor-solid (VS) mechanism on a Pt substrate over large scale (∼cm2), without the requirement of previous seed deposition. The grown ZnO NWs are single crystalline with wurtzite crystal structure and are vertically aligned respect to the growth substrate. Single NWs were then contacted by means of asymmetric contacts, with an electrochemically active and an electrochemically inert electrode, to form NW-based electrochemical metallization memory cells that show reproducible resistive switching behaviour and neuromorphic functionalities including short-term synaptic plasticity and paired pulse facilitation. Besides representing building blocks for NW-based memristive and neuromorphic systems, these single crystalline devices can be exploited as model systems to study physicochemical processing underlaying memristive functionalities thanks to the high localization of switching events on the ZnO crystalline surface.

Pure voltage-driven spintronic neuron based on stochastic magnetization switching behaviour

Voltage-driven stochastic magnetization switching in a nanomagnet has attracted more attention recently with its superiority in achieving energy-efficient artificial neuron. Here, a novel pure voltage-driven scheme with ∼27.66 aJ energy dissipation is proposed, which could rotate magnetization vector randomly using only a pair of electrodes covered on the multiferroic nanomagnet. Results show that the probability of 180° magnetization switching is examined as a sigmoid-like function of the voltage pulse width and magnitude, which can be utilized as the activation function of designed neuron. Considering the size errors of designed neuron in fabrication, it’s found that reasonable thickness and width variations cause little effect on recognition accuracy for MNIST hand-written dataset. In other words, the designed pure voltage-driven spintronic neuron could tolerate size errors. These results open a new way toward the realization of artificial neural network with low power consumption and high reliability.

Training and pattern recognition by an opto-magnetic neural network

Neuromorphic computing aims to mimic the architecture of the human brain to carry out computational tasks that are challenging and much more energy consuming for standard hardware. Despite progress in several fields of physics and engineering, the realization of artificial neural networks, which combine high operating speeds with fast and low-energy adaptability, remains a challenge. Here, we demonstrate an opto-magnetic neural network capable of learning and classification of digitized 3 × 3 characters exploiting local storage in the magnetic material. Using picosecond laser pulses, we find that micrometer sized synapses absorb well below 100 picojoule per synapse per laser pulse, with favorable scaling to smaller spatial dimensions. We, thus, managed to combine the speed and low-dissipation of optical networks with the low-energy adaptability and non-volatility of magnetism, providing a promising approach to fast and energy-efficient neuromorphic computing.

High-speed serial deep learning through temporal optical neurons.

Deep learning is able to functionally mimic the human brain and thus, it has attracted considerable recent interest. Optics-assisted deep learning is a promising approach to improve forward-propagation speed and reduce the power consumption of electronic-assisted techniques. However, present methods are based on a parallel processing approach that is inherently ineffective in dealing with the serial data signals at the core of information and communication technologies. Here, we propose and demonstrate a sequential optical deep learning concept that is specifically designed to directly process high-speed serial data. By utilizing ultra-short optical pulses as the information carriers, the neurons are distributed at different time slots in a serial pattern, and interconnected to each other through group delay dispersion. A 4-layer serial optical neural network (SONN) was constructed and trained for classification of both analog and digital signals with simulated accuracy rates of over 79.2% with proper individuality variance rates. Furthermore, we performed a proof-of-concept experiment of a pseudo-3-layer SONN to successfully recognize the ASCII codes of English letters at a data rate of 12 gigabits per second. This concept represents a novel one-dimensional realization of artificial neural networks, enabling a direct application of optical deep learning methods to the analysis and processing of serial data signals, while offering a new overall perspective for temporal signal processing.

Emulating artificial neuron and synaptic properties with SiO2-based memristive devices by tuning threshold and bipolar switching effects

The implementation of neuromorphic computations within a fully memristive neural network is considered the holy grail of the artificial intelligence era. In order to attain this goal, it is quite important to develop robust and configurable electronic devices capable of emulating spiking neuronal and synaptic plasticity activities. Along these lines, we report here the direct impact of oxygen concentration as well as of the homo-bilayer material configuration of SiO2-conductive bridge memories to the manifestation of tunable threshold and bipolar switching effects. Interestingly, while the bilayer structure of Ag/SiO x /SiO y /TiN (x < y) exhibits only bipolar switching effect, the respective single-layer structures of Ag/SiO y /TiN and Ag/SiO x /TiN operate under either threshold switching or both modes. Insights regarding the impact of oxygen concentration into the conducting filament growth process are provided. The manifestation of the two switching modes permits the emulation of various synaptic effects, such as short-term plasticity and long-term plasticity whereas the modulation of the conductance values allows the synaptic weight tuning by controlling the amplitude or the frequency of the triggering signals. Moreover, arbitrary neuron characteristics were obtained from our volatile memory devices without integrating any other auxiliary circuit. Our approach provides valuable insights towards the realization of artificial neural networks from the same material configuration with biological-like dynamic behavior.

Water-Mediated Ionic Migration in Memristive Nanowires with a Tunable Resistive Switching Mechanism.

Memristive devices based on electrochemical resistive switching effects have been proposed as promising candidates for in-memory computing and for the realization of artificial neural networks. Despite great efforts toward understanding the nanoionic processes underlying resistive switching phenomena, comprehension of the effect of competing redox processes on device functionalities from the materials perspective still represents a challenge. In this work, we experimentally and theoretically investigate the concurring reactions of silver and moisture and their impact on the electronic properties of a single-crystalline ZnO nanowire (NW). A decrease in electronic conductivity due to surface adsorption of moisture is observed, whereas, at the same time, water molecules reduce the energy barrier for Ag+ ion migration on the NW surface, facilitating the conductive filament formation. By controlling the relative humidity, the ratio of intrinsic electronic conductivity and surface ionic conductivity can be tuned to modulate the device performance. The results achieved on a single-crystalline memristive model system shed new light on the dual nature of the mechanism of how moisture affects resistive switching behavior in memristive devices.

An Artificial Neural Network Processor With a Custom Instruction Set Architecture for Embedded Applications

This article presents the design and implementation of an embedded programmable processor with a custom instruction set architecture for efficient realization of artificial neural networks (ANNs). The ANN processor architecture is scalable, supporting an arbitrary number of layers and number of artificial neurons (ANs) per layer. Moreover, the processor supports ANNs with arbitrary interconnect structures among ANs to realize both feed-forward and dynamic recurrent networks. The processor architecture is customizable in which the numerical representation of inputs, outputs, and signals among ANs can be parameterized to an arbitrary fixed-point format. An ASIC implementation of the designed programmable ANN processor for networks with up to 512 ANs and 262,000 interconnects is presented and is estimated to occupy 2.23 mm2 of silicon area and consume 1.25 mW of power from a 1.6 V supply while operating at 74 MHz in a standard 32-nm CMOS technology. In order to assess and compare the efficiency of the designed ANN processor, we have designed and implemented a dedicated reconfigurable hardware architecture for the direct realization of ANNs. Characteristics and implementation results of the designed programmable ANN processor and the dedicated ANN hardware on a Xilinx Artix-7 field-programmable gate array (FPGA) are presented and compared using two benchmarks, the MNIST benchmark using a feed-forward ANN and a movie review sentiment analysis benchmark using a recurrent neural network.

Brain‐Inspired Structural Plasticity through Reweighting and Rewiring in Multi‐Terminal Self‐Organizing Memristive Nanowire Networks

Acting as artificial synapses, two‐terminal memristive devices are considered fundamental building blocks for the realization of artificial neural networks. Current memristive crossbar architectures demonstrate the implementation of neuromorphic computing paradigms, although they are unable to emulate typical features of biological neural networks such as high connectivity, adaptability through reconnection and rewiring, and long‐range spatio‐temporal correlation. Herein, self‐organizing memristive random nanowire (NW) networks with functional connectivity able to display homo‐ and heterosynaptic plasticity is reported thanks to the mutual electrochemical interaction among memristive NWs and NW junctions. In particular, it is shown that rewiring and reweighting effects observed in single NWs and single NW junctions, respectively, are responsible for structural plasticity of the network under electrical stimulation. Such biologically inspired systems allow a low‐cost realization of neural networks that can learn and adapt when subjected to multiple external stimuli, emulating the experience‐dependent synaptic plasticity that shape the connectivity and functionalities of the nervous system that can be exploited for hardware implementation of unconventional computing paradigms.

Forming-Less Compliance-Free Multistate Memristors as Synaptic Connections for Brain-Inspired Computing

Hardware realization of artificial neural networks (ANNs) requires analogue weights to be encoded into the device conductances via blind update and access operations, leveraging Kirchhoff’s circuit laws. However, most memristive solutions lag behind in this aspect due to numerous device nonidealities, like limited number of addressable states, need for a stringent compliance current control, and an electroforming process. By modulating the oxygen vacancy profile of tin oxide switching elements, here we design and evaluate multistate memristors as synaptic connections for brain-inspired computing. Harnessing the advantages of a forming-less compliance-free operation, our devices display gradual switching transitions across multiple conductance states, sufficing the switching requirements of synaptic connections in an ANN. The soft boundary conditions are analyzed systematically, and spike-based plasticity rules, state-dependent spike-timing-dependent-plasticity (STDP) modulations, ternary digital logic, and analogue updatability schemes are proposed and demonstrated comprehensively to establish the analogue programming window of our memristors.

Developing near-infrared quantum-dot light-emitting diodes to mimic synaptic plasticity

The quantum-dot light-emitting diodes (QLEDs) that emit near-infrared (NIR) light may be important optoelectronic synaptic devices for the realization of artificial neural networks with complete optoelectronic integration. To improve the performance of NIR QLEDs, we take advantage of their low-energy light emission to explore the use of poly(3-hexylthiophene) (P3HT) as the hole transport layer (HTL). P3HT has one of the highest hole mobilities among organic semiconductors and essentially does not absorb NIR light. The usage of P3HT as the HTL indeed significantly mitigates the imbalance of carrier injection in NIR QLEDs. With the additional incorporation of an interlayer of poly [9,9-bis(3ʹ-( N , N -dimethylamino)propyl)-2,7-flourene]- alt -2,7-(9,9-dioctylfluorene)], P3HT obviously improves the performance of NIR QLEDs. As electroluminescent synaptic devices, these NIR QLEDs exhibit important synaptic functionalities such as short- and long-term plasticity, and may be employed for image recognition.

Organic memristive devices for perceptron applications

One of the most challenging tasks in neuromorphic applications, in the field of artificial intelligence, is the hardware realization of artificial neural networks (ANNs) which are able to learn during information processing (pattern recognition and classification, approximation, prediction, etc).In this scenario, thanks to their ability to keep the memory of their previous conductive state, memristors are widely considered as promising elements for the efficient implementation of ANNs. In this paper we present a short review of the design and the hardware realization of single and double layer ANNs, which are able to perform linearly separable and non-separable logic classification, using organic memristive devices as elements, ensuring the network weight adjustment.

The Realization of Artificial Neural Networks in Nonpositional Value for Automated Control Systems

The Realization of Artificial Neural Networks in Nonpositional Value for Automated Control Systems

Hardware Realization of Artificial Neural Network Based Intrusion Detection &amp;amp; Prevention System

In the 21st century with the exponential growth of the Internet, the vulnerability of the network which connects us is on the rise at a very fast pace. Today organizations are spending millions of dollars to protect their sensitive data from different vulnerabilities that they face every day. In this paper, a new methodology towards implementing an Intrusion Detection & Prevention System (IDPS) based on Artificial Neural Network (ANN) onto Field Programmable Gate Array (FPGA) is proposed. This system not only detects different network attacks but also prevents them from being propagated. The parallel structure of an ANN makes it potentially fast for the computation of certain tasks. FPGA platforms are the optimum and best choice for the modern digital systems nowadays. The same feature makes ANN well suited for implementation in FPGA technology. Hardware realization of ANN to a large extent depends on the efficient implementation of a single neuron. However FPGA realization of ANNs with a large number of neurons is still a challenging task. The proposed multilayer ANN based IDPS uses multiple neurons for higher performance and greater accuracy. Simulation of the design in MATLAB SIMULINK 2010b by using Knowledge Discovery and Data Mining (KDD) CUP dataset shows a very good performance. Subsequently MATLAB HDL coder was used to generate VHDL code for the proposed design that produced Intellectual Property (IP) cores for Xilinx Targeted Design Platforms. For evaluation purposes the proposed design was synthesized, implemented and tested onto Xilinx Virtex-7 2000T FPGA device.

Artificial neural network based on SQUIDs: demonstration of network training and operation

We propose a scheme for the realization of artificial neural networks based on superconducting quantum interference devices (SQUIDs). In order to demonstrate the operation of this scheme we designed and successfully tested a small network that implements an XOR gate and is trained by means of examples. The proposed scheme can be particularly convenient as support for superconducting applications such as detectors for astrophysics, high energy experiments, medicine imaging and so on.

VLSI Realization of Artificial Neural Networks with Improved Fault Tolerance

VLSI Realization of Artificial Neural Networks with Improved Fault Tolerance