Brain-inspired Computing Research Articles

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks.

The massively parallel nature of biological information processing plays an important role due to its superiority in comparison to human-engineered computing devices. In particular, it may hold the key to overcoming the von Neumann bottleneck that limits contemporary computer architectures. Physical-model neuromorphic devices seek to replicate not only this inherent parallelism, but also aspects of its microscopic dynamics in analog circuits emulating neurons and synapses. However, these machines require network models that are not only adept at solving particular tasks, but that can also cope with the inherent imperfections of analog substrates. We present a spiking network model that performs Bayesian inference through sampling on the BrainScaleS neuromorphic platform, where we use it for generative and discriminative computations on visual data. By illustrating its functionality on this platform, we implicitly demonstrate its robustness to various substrate-specific distortive effects, as well as its accelerated capability for computation. These results showcase the advantages of brain-inspired physical computation and provide important building blocks for large-scale neuromorphic applications.

Human action recognition with a large-scale brain-inspired photonic computer

The recognition of human actions in video streams is a challenging task in computer vision, with cardinal applications in e.g. brain-computer interface and surveillance. Deep learning has shown remarkable results recently, but can be found hard to use in practice, as its training requires large datasets and special purpose, energy-consuming hardware. In this work, we propose a scalable photonic neuro-inspired architecture based on the reservoir computing paradigm, capable of recognising video-based human actions with state-of-the-art accuracy. Our experimental optical setup comprises off-the-shelf components, and implements a large parallel recurrent neural network that is easy to train and can be scaled up to hundreds of thousands of nodes. This work paves the way towards simply reconfigurable and energy-efficient photonic information processing systems for real-time video processing.

Learning Algorithms and Signal Processing for Brain-Inspired Computing [From the Guest Editors

The articles in this special section focuses on machine learning (ML) and signal processing algorithms for bio-inspired computing. The articles bring together key researchers in this area to provide readers of IEEE Signal Processing Magazine with up-to-date and survey-style articles on algorithmic, hardware, and neuroscience perspectives on the state-of- the-art aspects of this emerging field.

Building brain-inspired computing

Building brain-inspired computing

Bipolar resistive switching, synaptic plasticity and non-volatile memory effects in the solution-processed zinc oxide thin film

The memristive devices are getting a lot of interest in recent years due to its applications in the field of resistive memory and brain-inspired computing. In the present work, we have developed a ZnO memristive device using the reflux method. We have demonstrated non-volatile memory properties and mimicked the basic synaptic properties of using Al/ZnO/FTO thin film device. The structural, morphological and compositional studies of ZnO thin film were carried out by X-ray diffraction, scanning electron microscopy and X-ray photoelectron spectroscopy, respectively. The co-existence of analog and digital resistive switching in a ZnO memristive device was achieved by properly tuning the external electrical stimulus. The developed ZnO memristive device mimics the basic synaptic properties such as potentiation-depression, symmetric Hebbian and antisymmetric Hebbian learning rules at a lower voltage bias (±1 V to ± 3 V). Whereas, non-volatile memory properties (endurance and retention) were achieved at a higher voltage bias (±4 V to ± 6 V). The Al/ZnO/FTO thin film memory device shows good resistive switching memory properties such as 104 endurance cycles and 104 s retention period with good uniformity during the cycle to cycle operation. The detailed analysis of I–V results suggested that the Schottky conduction model is responsible for the analog mode of operation, whereas, space charge limited current governs the device dynamics during the digital mode of operation. A possible resistive switching model is also presented.

Nano-scale single layer TiO2-based artificial synaptic device

Synaptic nano-electronic devices for brain-inspired computing have become increasingly popular because of their biological neuron-like properties such as massive parallelism with lower power consumption. Metal oxide-based resistive switching memory devices for the implementation of synapses are of great interest due to their low cost, easy production and complementary metal-oxide semiconductors (CMOS) compatibility. This study presents a simple, single-layer nano-scale TiO2-based artificial synaptic device for neuromorphic applications. The structural properties of the proposed nano-scale TiO2-based device were confirmed via both X-ray photoelectron spectroscopy (XPS) and energy dispersive X-ray spectroscopy (EDX). The bipolar resistive switching behavior of the device is shown by gradual increases in the low resistance (SET) state and gradual decreases in the high resistance (RESET) state. The synaptic characteristics of the device were resolved by applying voltage pulses. The typical potentiation and depression functions were obtained. Conforming to spike time dependent plasticity (STDP) was also achieved using synaptic weight changes. Homogeneous synaptic behaviors were associated with oxygen vacancies in the TiO2 layer, while abrupt changes in synaptic behavior were ascribed to filamentary transitions resulting from impurities in the metal oxide layer.

Highly Reliable Electronic Synapse Based on Au@Al2O3 Core-Shell Nanoparticles for Neuromorphic Applications

Simulating synaptic function and building brain-inspired computers have received widespread attention. The memristor is considered to be an electronic version of the synapse and an excellent element for creating a brand-new artificial intelligence system in the future. However, most memristor-based electronic synapses currently exhibit low stability due to complex resistance switching processes. Based on a controllable in-situ formation strategy that we were able to employ with atomic layer deposition, we fabricated electronic synapses based on Au@Al2O3 core-shell nanoparticles. Our device exhibited extremely reliable, stable, and durable performance and showed a capability to emulate biological synaptic behavior, including synaptic plasticity, long-term depression (LTD), long-term potentiation (LTP), amplitude dependence, and frequency dependence. This research provides a strategy for manufacturing highly-reliable electronic synapses for neuromorphic applications.

Design Space Exploration of Hardware Spiking Neurons for Embedded Artificial Intelligence

Machine learning is yielding unprecedented interest in research and industry, due to recent success in many applied contexts such as image classification and object recognition. However, the deployment of these systems requires huge computing capabilities, thus making them unsuitable for embedded systems. To deal with this limitation, many researchers are investigating brain-inspired computing, which would be a perfect alternative to the conventional Von Neumann architecture based computers (CPU/GPU) that meet the requirements for computing performance, but not for energy-efficiency. Therefore, neuromorphic hardware circuits that are adaptable for both parallel and distributed computations need to be designed. In this paper, we focus on Spiking Neural Networks (SNNs) with a comprehensive study of neural coding methods and hardware exploration. In this context, we propose a framework for neuromorphic hardware design space exploration, which allows to define a suitable architecture based on application-specific constraints and starting from a wide variety of possible architectural choices. For this framework, we have developed a behavioral level simulator for neuromorphic hardware architectural exploration named NAXT. Moreover, we propose modified versions of the standard Rate Coding technique to make trade-offs with the Time Coding paradigm, which is characterized by the low number of spikes propagating in the network. Thus, we are able to reduce the number of spikes while keeping the same neuron’s model, which results in an SNN with fewer events to process. By doing so, we seek to reduce the amount of power consumed by the hardware. Furthermore, we present three neuromorphic hardware architectures in order to quantitatively study the implementation of SNNs. One of these architectures integrates a novel hybrid structure: a highly-parallel computation core for most solicited layers, and time-multiplexed computation units for deeper layers. These architectures are derived from a novel funnel-like Design Space Exploration framework for neuromorphic hardware.

Gaussian synapses for probabilistic neural networks

The recent decline in energy, size and complexity scaling of traditional von Neumann architecture has resurrected considerable interest in brain-inspired computing. Artificial neural networks (ANNs) based on emerging devices, such as memristors, achieve brain-like computing but lack energy-efficiency. Furthermore, slow learning, incremental adaptation, and false convergence are unresolved challenges for ANNs. In this article we, therefore, introduce Gaussian synapses based on heterostructures of atomically thin two-dimensional (2D) layered materials, namely molybdenum disulfide and black phosphorus field effect transistors (FETs), as a class of analog and probabilistic computational primitives for hardware implementation of statistical neural networks. We also demonstrate complete tunability of amplitude, mean and standard deviation of the Gaussian synapse via threshold engineering in dual gated molybdenum disulfide and black phosphorus FETs. Finally, we show simulation results for classification of brainwaves using Gaussian synapse based probabilistic neural networks.

Multi-spectral gate-triggered heterogeneous photonic neuro-transistors for power-efficient brain-inspired neuromorphic computing

For the realization of low-power consumption brain-inspired neuromorphic computing devices which mimic the biological neuronal information processing methodology, the development of photonic transistors capable of synaptic behaviors and neuronal computation have attracted lots of interests. Here, metal-chalcogenide (MC)/metal-oxide (MO) heterogeneous photonic neuro-transistors capable of multi-spectrum triggered synaptic responses and corresponding neuronal computation were developed for an intelligent and energy efficient neuromorphic device. The photonic transistor architecture including a solution-processed broadband photo-active heterogeneous channel and electronic modulatory terminal enable to establish power-saved multi-level writing/reading processing. The multi-spectral gate-triggerings and their synaptic responses were emulated via the broadband absorbing MC/MO heterogeneous semiconducting structure and its defective hetero-interface, which can be fine-tuned by varying photo-spectrum of applied spikes and controlling of interfacial traps in-between, respectively. More importantly, the multi-spectrum triggered heterogeneous photonic neuro-transistors can facilitate wider dynamic and more intelligent neuronal computation such as multi-level dendritic summation and fire behaviors, logic-computation, and associated learning beyond conventional simple synaptic-level photonic devices. The results reported here argue that the multi-spectral activated heterogeneous photonic neuro-transistor outperforms current state-of-neuro-devices, provide a facile and generic route to achieve high-density and energy efficient neuromorphic system.

Fast Spiking of a Mott VO2-Carbon Nanotube Composite Device.

The recent surge of interest in brain-inspired computing and power-efficient electronics has dramatically bolstered development of computation and communication using neuron-like spiking signals. Devices that can produce rapid and energy-efficient spiking could significantly advance these applications. Here we demonstrate direct current or voltage-driven periodic spiking with sub-20 ns pulse widths from a single device composed of a thin VO2 film with a metallic carbon nanotube as a nanoscale heater, without using an external capacitor. Compared with VO2-only devices, adding the nanotube heater dramatically decreases the transient duration and pulse energy, and increases the spiking frequency, by up to 3 orders of magnitude. This is caused by heating and cooling of the VO2 across its insulator-metal transition being localized to a nanoscale conduction channel in an otherwise bulk medium. This result provides an important component of energy-efficient neuromorphic computing systems and a lithography-free technique for energy-scaling of electronic devices that operate via bulk mechanisms.

Synaptic behaviors of electromigrated Au nanogaps

Artificial electronic synapses or synaptic devices, which are capable of mimicking the functions of biological synapses in the human brain, are considered the basic building blocks for brain-inspired computing. Therefore, we investigated the emulation of synaptic functions in a simple Au nanogap. The synaptic functionality of neuromorphic hardware originates from a gradually modulated resistance. Previously, we investigated simple electromigration-based methods for controlling the tunnel resistance of nanogaps, called activation. In this study, a new type of artificial synaptic device based on planar Au nanogaps is demonstrated using a newly investigated activation procedure with voltage pulses. In the activation method with specific voltage pulses, the change in tunnel resistance of the Au nanogaps can be gradually controlled depending on the interval and amplitude of input voltage pulses. Moreover, Au inorganic synapses can emulate the synaptic functions of both short-term plasticity (STP) and long-term plasticity (LTP) characteristics. After the applied pulse is removed, the current decays rapidly at the beginning, followed by a gradual fade to a stable level. In addition, with repeated stimulations, the forgetting rate becomes decreases and the memory retention increases. Therefore, we observe an effect analogous to a memory transition from STP to LTP in biological systems. Our results may contribute to the development of highly functional artificial synapses and the further construction of neuromorphic computing architecture.

Flexible Transparent Organic Artificial Synapse Based on the Tungsten/Egg Albumen/Indium Tin Oxide/Polyethylene Terephthalate Memristor.

As artificial synapses in biomimetics, memristors have received increasing attention because of their great potential in brain-inspired neuromorphic computing. The use of biocompatible and degradable materials as the active resistive layer is promising in memristor fabrication. In this work, we select egg albumen as the resistive layer to fabricate flexible tungsten/egg albumen/indium tin oxide/polyethylene terephthalate devices, which can operate normally under mechanical bending without significant performance degradation. This proposed memristor device exhibits a transparency of more than 90% under visible light with a wavelength range of 230-850 nm. Moreover, by changing amplitudes of pulse voltage instead of intervals, paired-pulse facilitation can be transmitted to paired-pulse depression, which can faithfully mimic dynamical balance of Ca2+ concentration shaped by voltage-sensitive calcium channels. The device resistance can be modulated gradually by applied pulse trains to mimic certain neural bionic behaviors, including excitatory postsynaptic current, short-term plasticity (STP) and long-term plasticity (LTP), and transitions between STP and LTP. The reasons behind these behaviors are analyzed through power consumption calculation. Excellent biosimulation characteristics have been demonstrated in this egg albumen-based memristor device, which is desirable in biocompatible and dissolvable electronics for flexible artificial neuromorphic systems.

Organic Synapses for Neuromorphic Electronics: From Brain-Inspired Computing to Sensorimotor Nervetronics.

Living organisms have a long evolutionary history that has provided them with functions and structures that enable them to survive in their environment. The goal of biomimetic technology is to emulate these traits of living things. Research in bioinspired electronics develops electronic sensors and motor systems that mimic biological sensory organs and motor systems and that are intended to be used in bioinspired applications such as humanoid robots, exoskeletons, and other devices that combine a living body and an electronic device. To develop bioinspired robotic and electronic devices that are compatible with the living body at the neuronal level and that are operated by mechanisms similar to those in a living body, researchers must develop biomimetic electronic sensors, motor systems, brains, and nerves. Artificial organic synapses have emulated the brain's plasticity with much simpler structures and lower fabrication cost than neurons based on silicon circuits, and with smaller energy consumption than traditional von Neumann computing methods. Organic synapses are promising components of future neuromorphic systems. In this Account, we review recent research trends of neuromorphic systems based on organic synapses, then suggest research directions. We introduce the device structures and working mechanisms of reported organic synapses and the brain's plasticity, which are mainly imitated to demonstrate the learning and memory function of the organic synapses. We also introduce recent reports on sensory synapses and sensorimotor nervetronics that mimic biological sensory and motor nervous systems. Sensory nervetronics can be used to augment the sensory functions of the living body and to comprise the sensory systems of biomimetic robots. Organic synapses can also be used to control biological muscles and artificial muscles that have the same working mechanism as biological muscle. Motor nervetronics would impart life-like motion to bioinspired robots. Chemical approaches may provide insights to guide development of new organic materials, device structures, and working mechanisms to improve synaptic responses of organic neuromorphic systems. For example, organic synapses can be applied to electronic and robotic skins and bioimplantable medical devices that use mechanically stable, self-healing, and biocompatible organic materials. Biochemical approaches may expand the plasticity of the brain and nervous system. We expect that organic neuromorphic systems will be vital components in bioinspired robotic and electronic applications, including biocompatible neural prosthetics, exoskeletons, humanoid soft robots, and cybernetics devices that are integrated with biological and artificial organs.

Memristive crossbar arrays for brain-inspired computing.

With their working mechanisms based on ion migration, the switching dynamics and electrical behaviour of memristive devices resemble those of synapses and neurons, making these devices promising candidates for brain-inspired computing. Built into large-scale crossbar arrays to form neural networks, they perform efficient in-memory computing with massive parallelism by directly using physical laws. The dynamical interactions between artificial synapses and neurons equip the networks with both supervised and unsupervised learning capabilities. Moreover, their ability to interface with analogue signals from sensors without analogue/digital conversions reduces the processing time and energy overhead. Although numerous simulations have indicated the potential of these networks for brain-inspired computing, experimental implementation of large-scale memristive arrays is still in its infancy. This Review looks at the progress, challenges and possible solutions for efficient brain-inspired computation with memristive implementations, both as accelerators for deep learning and as building blocks for spiking neural networks.

IGZO/Al2O3 based depressed synaptic transistor

Bionic devices, such as potentiated synaptic devices, have obtained a lot of attention over the past years. However, depressed synaptic devices, which are equally significant in brain-inspired computation devices, have attracted less attention due to the lack of a proper electrolyte. In this work, an indium-gallium-zinc-oxide (IGZO) and aluminum oxide (Al2O3) film based synaptic transistor was fabricated for depressed neuromorphic applications. An Al2O3 film desposited by the atomic layer deposition (ALD) at 100 °C was used as the insulator. The low temperature introduced a lot of electron trappings at the IGZO-Al2O3 interface, which contributed to the depression feature of the synaptic transistor. The energy consumption of the fabricated transistor was estimated to be as low as ∼3.8 pJ/spike. The recovery time of a pulse was as short as ∼150 ms. Paired-pulse depression (PPD) and low-frequency filter behaviors were also mimicked in the transistor.

Nano Resistive Memory (Re-RAM) Devices and their Applications

Abstract Use of solid state ionic conductors the so-called Solid Electrolytes has brought new impetus to the field of solid state memories namely resistive random access memory (Re-RAM). In this review article, to begin we present the detailed understanding on the basics of solid electrolytes. Later, the same has been reviewed focusing on its application in novel solid state memory applications. Few examples of solid electrolytes are considered and their impact on the state-of-art research in this domain is discussed in detail. An in-depth analysis on the fundamentals of Resistive switching mechanism involved in various classes of Memristive devices viz., Electrochemical Metallization Memories (ECM) and Valence change Memories (VCM). A few important applications of Memristors such as Neuristor and artificial synapse in neuromorphic computing are reviewed as well. Finally, the most anticipated energy efficient battery-like cells as artificial synapse in brain-inspired computing is also covered.

Fabrication of synaptic memristor based on two-dimensional material MXene and realization of both long-term and short-term plasticity

Compared with conventional computation relying on the von Neumann architecture, brain-inspired computing has shown superior strength in various cognitive tasks. It has been generally accepted that information in the brain is represented and formed by vastly interconnected synapses. So the physical implementation of electronic synaptic devices is crucial to the development of brain-based computing systems. Among a large number of electronic synaptic devices, the memristors have attracted significant attention due to its simple structure and similarities to biological synapses. In this work, we first use two-dimensional material MXene as a resistive material and fabricate an electronic synapse based on a Cu/MXene/SiO<sub>2</sub>/W memristor. By using the unique properties of MXene, the conductance of the memristor can be modulated by the accumulation or reflux of Cu<sup>2+</sup> at the physical switching layer, which can vividly simulate the mechanism of bio-synapses. Experimental results show that the Cu/MXene/SiO<sub>2</sub>/W memristor not only achieves stable bipolar analog resistance switching but also shows excellent long-term and short-term synaptic behaviors, including paired-pulse facilitation (PPF) and long-term potential/depression. By adjusting the pulse interval, the PPF index will change accordingly. In a biological system, the short-term plasticity is considered to be the key point for performing computational functions while the long-term plasticity is believed to underpin learning and memory functions. This work indicates that Cu/MXene/SiO<sub>2</sub>/W memristor with both long-term and short-term plasticity will have great application prospects for brain-inspired intelligence in the future.

Synapse behavior characterization and physical mechanism of a TiN/SiOx/p-Si tunneling memristor device

The demand for massive deep learning neural networks has driven the development of nanoscale memristor devices, which perform brain-inspired neuromorphic computing.

A versatile neuromorphic system based on simple neuron model

Brain-inspired neuromorphic computing has attracted much attention for its advanced computing concept. However, the massive hardware cost in fully-connected architectures makes it challenging to build a large-scale neuromorphic system. In this work, we report a compact, programmable, versatile, and scalable neuromorphic architecture. To demonstrate the concept of the neuromorphic architecture, a neuromorphic system consisting of four cores is implemented on an FPGA platform. On the one hand, the neuromorphic system is extremely compact and hardware-saving. The computing block based on a simple digital leaky Integrate-and-Fire (LIF) model only costs 69 logic elements (LEs); only one physical neuron is implemented in each core, and it can be reused as hundreds of virtual neurons by time-division-multiplexing; only four 9-bit synaptic weights are assigned to each neuron, which effectively alleviates the hardware explosion in fully-connected architecture. On the other hand, the neuromorphic system is programmable and versatile, and can perform different neural network computing. The neuromorphic system mapped with a three-layer feedforward network successfully recognizes the MNIST handwritten digits with an accuracy of 96.26%, and it also effectively realizes different convolution operations which are basic computing operations in convolutional neural networks. Last but not least, each neuromorphic core has its own router module, making it convenient to scale up.