Bio-inspired Neuromorphic Systems Research Articles

A reservoir computing system with volatile and non-volatile organic memristors as a promising hardware architecture

In recent years, many scientific groups have been working on hardware implementation of the artificial neural networks to approach the computational efficiency of their biological counterpart. Memristors may play the role of synapses in such networks [1]. Varieties of memristive structures and materials have already been tested in different neural network architectures, but still no memristor is considered ideal for hardware synapse implementation [1]. One of the most significant problems is the presence of inherent stochasticity distinctive for all memristive devices, which complicates the training of the neural networks [1]. Several approaches were proposed to partially mitigate this problem, e.g., a reservoir computing system (RCS) [2] and spiking neural networks (SNN) [3] as well as defect engineering for memristive characteristics improvement. In this work, we propose to combine RCS with SNN and create a bio-inspired neuromorphic system based on two types of organic memristors with specifically designed structures and advanced characteristics. The RCS consists of two main parts: the reservoir and the readout [2]. The reservoir layer extracts some representative features from the input data due to its internal nonlinear dynamics. The readout layer then uses these features to classify the input data. Typically, a conventional fully connected neural network is used as a readout layer in the RCS. The training process occurs only in the readout layer, while a reservoir is not trainable. This decrease in trainable parameters considerably reduces the memristive stochasticity impact on the training process. The use of different types of memristors for the RCS is essential. The reservoir layer should consist of memristors with short-term memory, i.e., volatile memristors. This way, memristors can process each input sample individually. Volatile polyaniline-based memristors were chosen for this layer implementation. They can operate within a biologically plausible time range, which is essential as we aim to mimic biological systems [4]. In contrast, the reservoir layer should consist of memristors with long-term memory, i.e., non-volatile memristors, because the readout layer should preserve the trained synaptic weights. Non-volatile parylene memristors with incorporated MoO3 nanoparticles were chosen for the readout layer. The reservoir computing system adopts some essential principles of brain function, as both short- and long-term memory are significant in biological systems. However, traditional neural networks are commonly used as a readout layer in the RCSs [2]. Their training requires global weight updates, making them vulnerable to memristive stochasticity. In contrast, the SNNs allow local training, e.g., using bio-inspired learning rules, which makes them more effective and robust [3]. Consequently, we presume that a fully organic RCS with an SNN readout layer is a promising hardware memristive architecture. The work consists of two parts: hardware and software. First, the polyaniline- and parylene-based memristive devices were fabricated and tested. Hardware polyaniline reservoir demonstrated an ability to extract characteristic features from the input data. Nanocomposite parylene memristors were suitable for the role of synapses in the readout layer due to the unique combination of high switching speed, high stability, low power consumption and the possibility of crossbar implementation. Next, the traditional and spiking readout layers were compared in simulation. It was shown that the SNN readout layer is more adaptive and sustainable to noise in image classification tasks as well as memristive stochasticity [5].

Realization of dual-functional resistive switching characteristics in Ag−In−Zn−S/sericin peptide-based memristive device

Employing suitable materials and device engineering is one of the crucial methods toward the realization of multifunctional memristive devices for constructing bioinspired neuromorphic systems. In this work, dual-functional memristors composed of eco-friendly natural silk sericin, coexistently enabling the achievement of threshold switching and memory switching triggered by adjusting the compliance current value, have been fabricated with a specific two-terminal device structure: Ag/Ag−In−Zn−S/silk sericin/W. Experimentally, the as-manufactured memristors exhibit several desirable qualities, such as low switching voltage (< 0.7 V), relatively small cycle-to-cycle and device-to-device variabilities, nonvolatile multilevel storage characteristics, and rapid switching speed (40 ns). Beyond these qualities, fundamental synaptic behaviors, such as paired-pulse facilitation and spike-timing-dependent plasticity (STDP), have been mimicked. This was made possible by a filamentary mechanism based on Ag migration. The fitted time constants corresponding to the STDP potentiation and depression are about 30 ms, and the highest changes in synaptic weight for positive and negative voltage pulses are 84.4% and 61.7%, respectively. Furthermore, the typical coincidence detection task has been executed, demonstrated by simulation based on the fitted STDP's parameters of the sericin-based device. The results from this study indicate that the sericin-based memristors, as designed, have the potential to be employed in the creation of versatile neuromorphic devices for neuromorphic computing systems.

Diverse long-term potentiation and depression based on multilevel LiSiO x memristor for neuromorphic computing

Memristor-based neuromorphic computing is expected to overcome the bottleneck of von Neumann architecture. An artificial synaptic device with continuous conductance variation is essential for implementing bioinspired neuromorphic systems. In this work, a memristor based on Pt/LiSiO x /TiN structure is developed to emulate an artificial synapse, which shows non-volatile multilevel resistance state memory behavior. Moreover, the high nonlinearity caused by abrupt changes in the set process is optimized by adjusting the initial resistance. 100 levels of continuously modulated conductance states are achieved and the nonlinearity factors are reduced to 1.31. The significant improvement is attributed to the decrease in the Schottky barrier height and the evolution of the conductive filaments. Finally, due to the improved linearity of the long-term potentiation/long-term depression behaviors in LiSiO x memristor, a robust recognition rate (∼94.58%) is achieved for pattern recognition with the modified National Institute of Standards and Technology handwriting database. The Pt/LiSiO x /TiN memristor shows significant potential in high-performance multilevel data storage and neuromorphic computing systems.

Emulation of bio-synaptic behaviours in copper-doped zinc oxide memristors: A nanoscale scanning probe microscopic study

Memristors emulating biological synapses to perform memory and learning functions are crucial for realizing bio-inspired neuromorphic systems. However, to meet the increasing demand for high-speed operation, low power consumption, and large integration density in neuromorphic architectures, the synaptic functions must be achieved at nanometer scale and with ultra-low write current. This report evidences electronic synapses realized in Cu-doped ZnO (CZO) memristors by nanoscale scanning probe microscopic techniques. In this process, the device can be downsized all the way up to the size of the probe, offering ultra-high density (>109 devices/mm2) of electronic synapses. The essential bio-synaptic functions in the memristor are accomplished with ultra-low write current (∼10 nA) and in a forming-free approach that marks their potential in building power-efficient computing systems. The modulation in device conductivity is elucidated on the basis of charged defect migration, forming conducting bridges across the thickness or locally within the CZO layer. In addition, the influence of Cu-doping in controlling nanoscale neuromorphic and memristive performances of the device is explored. The present study, therefore, demonstrates a novel approach to decipher oxide-based synaptic-memristors and highlights CZO as a promising candidate to fabricate ultra high-density power-efficient neuromorphic devices.

Efficient Resistive Switching and Spike Rate Dependent Plasticity in a New CuCrO2 Memristor for Plausible Neuromorphic Systems

In this article, we introduce a new class of p-type transparent conductive oxide (TCO) CuCrO2 (150 nm) heterogeneously integrated onto fluorine doped tin oxide (FTO)/glass for forming-free memristor-based neuromorphic applications. The fabricated Al/CuCrO2/FTO memristors demonstrate a reliable bipolar resistive switching with an ON/ OFF ratio of 1000. The retention of the device was found to be steady even beyond 106 s, which demonstrates its nonvolatility. The current–voltage ( ${I}$ – ${V}$ ) characteristics were fit to evaluate its transport properties and a band diagram was projected to have a better insight of the device operational principles. To validate the experimental observations, a new model has been developed, and the simulated ${I}$ – ${V}$ behavior was analogous to the experimental one. Efforts were then devoted to observe long-term potentiation (LTP) and long-term depression (LTD) utilizing identical but opposite pulses to evaluate the device’s efficacy for synaptic applications. The synaptic behavior was well controlled by the pulse (pulse amplitude and width) variations. The conductance change was found to be symmetric and then saturated, which reflects the popular biological Hebbian rules. Finally, a long-term synaptic modulation has been implemented by establishing the spike rate dependent plasticity (SRDP) rule, which is a part of spiking neural networks and advantageous to mimic the brain’s capability at low power. All the obtained experimental results were systematically corroborated by neural network simulation. Overall, our approach provides a new road map toward the development of TCO-based alternative memristors, which can be employed to mimic the synaptic plasticity for energy-efficient bioinspired neuromorphic systems and non-von Neumann computer architectures.

Solving Overlapping Pattern Issues in On-Chip Learning of Bio-Inspired Neuromorphic System with Synaptic Transistors

Recently, bio-inspired neuromorphic systems have been attracting widespread interest thanks to their energy-efficiency compared to conventional von Neumann architecture computing systems. Previously, we reported a silicon synaptic transistor with an asymmetric dual-gate structure for the direct connection between synaptic devices and neuron circuits. In this study, we study a hardware-based spiking neural network for pattern recognition using a binary modified National Institute of Standards and Technology (MNIST) dataset with a device model. A total of three systems were compared with regard to learning methods, and it was confirmed that the feature extraction of each pattern is the most crucial factor to avoiding overlapping pattern issues and obtaining a high pattern classification ability.

Fully transparent, flexible and waterproof synapses with pattern recognition in organic environments

Artificial intelligence applications require bio-inspired neuromorphic systems that consist of electronic synapses (e-synapses) able to perform learning and memory functions.

Flexible Electronic Synapses for Face Recognition Application with Multimodulated Conductance States.

An artificial synaptic device with a continuous weight modulation behavior is fundamental to the hardware implementation of the bioinspired neuromorphic systems. Recent reported synaptic devices have a less number of conductance states, which is not beneficial for the continuous modulation of weights in neuromorphic computing. Preparing a device with as many conductance states as possible is of great significance to the development of brain-inspired neuromorphic computing. Here, we present a two-terminal flexible organic synaptic device with ultra-multimodulated conductance states, realizing a face recognition functionality with a strong error-tolerant nature for the first time. The device shows an excellent long-term potentiation or long-term depression behavior and reliability after 1000 folded destructive tests. There are 600 continuous ultra-multimodulated conductance states, which can be used to realize the great face recognition capability. The recognition rates were 95.2% and above 90% for the initial and 15% noise pixel images, respectively. The strong error-tolerant nature indicates a potential application of a flexible organic artificial synaptic device with ultra-multimodulated conductance states in the large-scale neuromorphic systems.

An Organic Flexible Artificial Bio-Synapses with Long-Term Plasticity for Neuromorphic Computing.

Artificial synapses, with synaptic plasticity, are the key components of constructing the neuromorphic computing system and mimicking the bio-synaptic function. Traditional synaptic devices are based on silicon and inorganic materials, while organic electronics can open up new opportunities for flexible devices. Here, a flexible artificial synaptic device with an organic functional layer was proposed. The organic device showed good switching behaviors such as ON/OFF ratio over 100 at low operation voltages. The set and reset voltages were lower than 0.5 V and −0.25 V, respectively. The long-term plasticity, spike-timing-dependent plasticity learning rules (STDP), and forgetting function were emulated using the device. The retention times of the excitatory and inhibitory post-synaptic currents were both longer than 60 s. The long-term plasticity was repeatable without noticeable degradation after the application of five voltage pulse cycles to the top electrode. These results indicate that our organic flexible device has the potential to be applied in bio-inspired neuromorphic systems.

Artificial Synapses with Short- and Long-Term Memory for Spiking Neural Networks Based on Renewable Materials.

Emulation of biological synapses that perform memory and learning functions is an essential step toward realization of bioinspired neuromorphic systems. Artificial synaptic devices have been developed based mostly on inorganic materials and conventional semiconductor device fabrication processes. Here, we propose flexible biomemristor devices based on lignin by a simple solution process. Lignin is one of the most abundant organic polymers on Earth and is biocompatible, biodegradable, as well as environmentally benign. This memristor emulates several essential synaptic behaviors, including analog memory switching, short-term plasticity, long-term plasticity, spike-rate-dependent plasticity, and short-term to long-term transition. A flexible lignin-based artificial synapse device can be operated without noticeable degradation under mechanical bending test. These results suggest lignin can be a promising key component for artificial synapses and flexible electronic devices.

Pattern recognition with TiO<sub>x</sub>-based memristive devices

We report on the development of TiOx-based memristive devices for bio-inspired neuromorphic systems. In particular, capacitor like structures of Al/AlOx/TiOx/Al with, respectively 20 nm and 50 nm thick TiOx-layers were fabricated and analyzed in terms of their use in neural network circuits. Therefore, an equivalent circuit model is presented which mimics the observed device properties on a qualitative level and relies on mobile oxygen ions by taking electronic transport through local conducting filaments and hopping between TiOx defect states into account. The model also comprises back diffusion of oxygen ions and allows for a realistic description of the experimental recorded device characteristics. The in Refs. [1-3] reported computing paradigms for pattern recognition have been used as guidelines for a device performance investigation at the network level. In particular, simulations of a spiking neural network are presented which allows for pattern recognition. As input patterns hand written digits taken from the MNIST Data base have been used. Within the network the memristive devices are arranged in a cross-bar array connected by 196 input neurons and ten output neurons. While, each input neuron corresponds to a specific pixel of the image of the input pattern, the output neurons were implemented as spiking neurons. In addition, the output neurons were inhibitory linked within an winner-take-it-all network and consist of a homeostasis-like behavior for their spiking thresholds. Based on the network simulation essential requirements for the development of optimal memristive device for neuromorphic circuits are discussed.

Neuromorphic Visual Information Processing –Bio-inspired Vision

 Abstract—This paper describes the early vision of bio-inspired neuromorphic system, mimicking the primitive behaviour of visual cortex. The proposed bio-inspired vision exhibits the biologically plausible function of mimicking the cat's visual cortex experimentation of Hubel and Wiesel. The neuromorphic implementation of vision is inspired by the directional visual signal selectivity of cortex and the CMOS spiking neuron based on Hodgkin-Huxley formalism. The feasibility of neuromorphic early vision is demonstrated for applications of the transportation safety system of passengers' occupancy detection of 95%, based on the recognition of human head figures.