CMOS Neuron Research Articles

Building an Analog Circuit Synapse for Deep Learning Neuromorphic Processing

In this article, we propose a circuit to imitate the behavior of a Reward-Modulated spike-timing-dependent plasticity synapse. When two neurons in adjacent layers produce spikes, each spike modifies the thickness in the shared synapse. As a result, the synapse’s ability to conduct impulses is controlled, leading to an unsupervised learning rule. By introducing a reward signal, reinforcement learning is enabled by redirecting the growth and shrinkage of synapses based on signal feedback from the environment. The proposed synapse manages the convolution of the emitted spike signals to promote either the strengthening or weakening of the synapse, represented as the resistance value of a memristor device. As memristors have a conductance range that may differ from the available current input range of typical CMOS neuron designs, the synapse circuit can be adjusted to regulate the spike’s amplitude current to comply with the neuron. The circuit described in this work allows for the implementation of fully interconnected layers of neuron analog circuits. This is achieved by having each synapse reconform the spike signal, thus removing the burden of providing enough power from the neurons to each memristor. The synapse circuit was tested using a CMOS analog neuron described in the literature. Additionally, the article provides insight into how to properly describe the hysteresis behavior of the memristor in Verilog-A code. The testing and learning capabilities of the synapse circuit are demonstrated in simulation using the Skywater-130 nm process. The article’s main goal is to provide the basic building blocks for deep neural networks relying on spiking neurons and memristors as the basic processing elements to handle spike generation, propagation, and synaptic plasticity.

Capacitor-free Scalable CMOS Neuron Circuit with Compact Design and Low Power Consumption

Capacitor-free Scalable CMOS Neuron Circuit with Compact Design and Low Power Consumption

A 5.3 pJ/Spike CMOS Neural Array Employing Time-Modulated Axon-Sharing and Background Mismatch Calibration Techniques

Inspired by the human brain, spiking neuron networks are promising to realize energy-efficient and low-latency neuromorphic computing. However, even state-of-the-art silicon neurons are orders of magnitude worse than biological neurons in terms of area and power consumption due to the limitations. Moreover, limited routing in typical CMOS processes is another challenge for realizing the fully-parallel high-throughput synapse connections compared to biological synapses. This paper presents an SNN circuit that utilizes resource-sharing techniques to address the two challenges. Firstly, a comparator sharing neuron circuit with a background calibration technique is proposed to shrink the size of a single neuron without performance degradation. Secondly, a time-modulated axon-sharing synapse system is proposed to realize a fully-parallel connection with limited hardware overhead. To validate the proposed approaches, a CMOS neuron array is designed and fabricated under a 55-nm process. It consists of 48 LIF neurons with 3125 neurons/mm 2 area density, power consumption of 5.3 pJ/spike, and equivalent 2304 fully parallel synapses providing a unit throughput of 5500 events/s/neuron. It proves the proposed approaches are promising to realize a high-throughput high-efficiency SNN with CMOS technology.

Energy- and Area-Efficient CMOS Synapse and Neuron for Spiking Neural Networks With STDP Learning

This paper proposes CMOS synapse and neuron for use in spiking neural networks to perform cognitive functions in a bio-inspired manner. The proposed synapse can trace the eligibility of the timing relationship between pre- and post-synaptic spikes, supporting a bio-plausible local learning rule called the spike timing-dependent plasticity (STDP) in an energy- and area-efficient manner. The proposed neuron can support neural functions such as synaptic current integration, threshold-based firing, neuronal leaking, membrane potential resetting, and adjustable refractory period with improved energy and area efficiency. The STDP curve shape of the synapse and the firing rate of the neuron can be adjusted as desired. Their variability due to process, voltage, and temperature (PVT) variations can also be minimized. The proposed CMOS neuron and synapse circuits were designed in a 28-nm CMOS process. The performance evaluation results indicate that the proposed synapse reduces energy consumption and area by up to 94% and 43% compared to conventional CMOS synapses. They also indicate that the proposed neuron achieves energy and area reductions of 37% and 23%, respectively, compared to conventional CMOS neurons. An associative neural network composed of the proposed neuron and synapse was designed to verify that they together work well for performing a cognitive function of associative learning and inferencing.

Impact of the leakage current of an AND-type synapse array on spiking neural networks

In this paper, we analyze the impact of the leakage current of an AND-type synapse array on a spiking neural network (SNN). In order to evaluate the firing rate modulation affected by the leakage current of a synapse array, the electrical characteristics of a fabricated dual-gate synapse are measured and modeled. A SPICE circuit simulation shows a severe change in the firing rate in the SNN, and a system-level simulation with MNIST datasets exhibits frequency-dependent performance degradation due to the leakage current. Compared to a case with negligible leakage current, a 6.98% accuracy loss at 50 kHz of input frequency, which is the firing rate of conventional CMOS neurons.

A CMOS-memristor hybrid system for implementing stochastic binary spike timing-dependent plasticity.

This paper describes a fully experimental hybrid system in which a memristive crossbar spiking neural network (SNN) was assembled using custom high-resistance state memristors with analogue CMOS neurons fabricated in 180 nm CMOS technology. The custom memristors used NMOS selector transistors, made available on a second 180 nm CMOS chip. One drawback is that memristors operate with currents in the micro-amperes range, while analogue CMOS neurons may need to operate with currents in the pico-amperes range. One possible solution was to use a compact circuit to scale the memristor-domain currents down to the analogue CMOS neuron domain currents by at least 5–6 orders of magnitude. Here, we proposed using an on-chip compact current splitter circuit based on MOS ladders to aggressively attenuate the currents by over 5 orders of magnitude. This circuit was added before each neuron. This paper describes the proper experimental operation of an SNN circuit using a 1T1R synaptic crossbar together with four post-synaptic CMOS circuits, each with a 5-decade current attenuator and an integrate-and-fire neuron. It also demonstrates one-shot winner-takes-all training and stochastic binary spike-timing-dependent-plasticity learning using this small system.This article is part of the theme issue ‘Advanced neurotechnologies: translating innovation for health and well-being’.

Review of Analog Neuron Devices for Hardware-based Spiking Neural Networks

To process data operations more efficiently in deep neural networks (DNNs), studies on spiking neural networks (SNNs) have been conducted. In the reported literature, CMOS neuron circuits that mimic the biological behavior of an integrate-and-fire function of neurons have been mainly studied. Because conventional neuronal circuits need to be improved in terms of area and energy consumption, neuron devices with memory functions such as resistive random access memory (RRAM), phase-change random access memory (PCRAM), magnetic random access memory (MRAM), floating body FETs, and ferroelectric FETs have been emerged to replace a membrane capacitor and trigger device in the conventional neuron circuits. In this review article, neuron devices that can increase the integration density of conventional neuronal circuits and reduce power consumption are reviewed. These devices are expected to play an important role in future neuromorphic systems.

An Ultra-Low Power Threshold Voltage Variable Artificial Retina Neuron

An artificial retina neuron is proposed and implemented by CMOS technology. It can be used as an image sensor in the Artificial Intelligence (AI) field with the benefit of ultra-low power consumption. The artificial neuron can generate signals in spike shape with pre-designed frequencies under different light intensities. The power consumption is reduced by removing the film capacitor. The comparator is adopted to improve the stability of the circuit, and the power consumption of the comparator is optimized. The power consumption of the proposed CMOS neuron circuit is suppressed. The ultra-low-power artificial neuron with variable threshold shows a frequency range of 0.8–80 kHz when the input current is varied from 1 pA to 150 pA. The minimum DC power is 35 pW when the input current is 5 pA. The minimum energy of the neuron is 3 fJ. The proposed ultra-low-power artificial retina neuron has wide potential applications in the field of AI.

MESO Neuron: A Low-Power and Ultrafast Spin Neuron for Neuromorphic Computing

In this letter, a low-power and ultrafast spin neuron for mimicking biological neurons based on magneto-electric spin-orbit (MESO) neurons is presented. First, the physical model of a MESO neuron based on the Landau–Lifshitz–Gilbert (LLG) equation at room temperature is built for investigating the characteristics. By utilizing these characteristics of the MESO device, a current pulse is used to induce the stochastic switching behaviors. We successfully mimic the behavior of the biological neuron with single activation time down to 0.8 ns. Second, using model-derived device parameters, we further simulate a three-layer fully connected neural network using MESO neurons. Using the Mixed National Institute of Standards and Technology database handwritten pattern dataset, our system achieves a recognition accuracy of 98%. In addition, the influence of pulsewidth and amplitude on activation functions of MESO neurons is researched using HSPICE tools. The results show that as pulsewidth and amplitude are increasing, the power consumption and computing time increase while the energy consumption decreases. Specifically, the power consumption performance of a MESO neuron is about 10 µW and improved approximately three orders of magnitude compared to a 45 nm CMOS neuron.

Activation function and computing performance of spin neuron driven by magnetic field and strain

The spin neuron is an emerging artificial neural device which has many advantages such as ultra-low power consumption, strong nonlinearity, and high integration. Besides, it has ability to remember and calculate at the same time. So it is seen as a suitable and excellent candidate for the new generation of neural network. In this paper, a spin neuron driven by magnetic field and strain is proposed. The micromagnetic model of the device is realized by using the OOMMF micromagnetic simulation software, and the numerical model of the device is also established by using the LLG equation. More importantly, a three-layer neural network is composed of spin neurons constructed respectively using three materials (Terfenol-D, FeGa, Ni). It is used to study the activation functions and the ability to recognize the MNIST handwritten datasets.c Results show that the spin neuron can successfully achieve the random magnetization switching to simulate the activation behavior of the biological neuron. Moreover, the results show that if the ranges of the inputting magnetic fields are different, the three materials' neurons can all reach the saturation accuracy. It is expected to replace the traditional CMOS neuron. And the overall power consumption of intelligent computing can be further reduced by using appropriate materials. If we input the magnetic fields in the same range, the recognition speed of the spin neuron made of Ni is the slowest in the three materials. The results can establish a theoretical foundation for the design and the applications of the new artificial neural networks and the intelligent circuits.

Efficient Hybrid CMOS/Memristor Implementation of Bidirectional Associative Memory Using Passive Weight Array

The primacy of memristors for the realization of the synaptic circuit makes this element a promising component for implementing Artificial Neural Networks (ANN). However, integrating memristors with other ANN components is still an open area of research. Therefore, a circuit-level hybrid CMOS/Memristor architecture for implementing Bidirectional Associative Memory (BAM) is proposed in this paper. A passive bi-directional pre-differentiation crossbar architecture has been utilized and applied for the realization of synaptic weights. A compact, low voltage CMOS neuron that consumes a low amount of power is also designed. The noise tolerance of the proposed hardware is evaluated for typical networks with different sizes and a number of stored patterns. It is demonstrated that the hardware pattern retrieval rate is similar to its software-based counterpart. The influence of memristor non-idealities on BAM is also investigated for the first time, and a practical method has been introduced to deal with this issue. In comparison to the available circuit-level hardware implementation of memristive associative memories, the proposed circuit consumes significantly less power and retrieves the stored patterns notably faster.

Low Power, CMOS-MoS2 Memtransistor based Neuromorphic Hybrid Architecture for Wake-Up Systems

Neuromorphic architectures have become essential building blocks for next-generation computational systems, where intelligence is embedded directly onto low power, small area, and computationally efficient hardware devices. In such devices, realization of neural algorithms requires storage of weights in digital memories, which is a bottleneck in terms of power and area. We hereby propose a biologically inspired low power, hybrid architectural framework for wake-up systems. This architecture utilizes our novel high-performance, ultra-low power molybdenum disulphide (MoS2) based two-dimensional synaptic memtransistor as an analogue memory. Furthermore, it exploits random device mismatches to implement the population coding scheme. Power consumption per CMOS neuron block was found to be 3 nw in the 65 nm process technology, while the energy consumption per cycle was 0.3 pJ for potentiation and 20 pJ for depression cycles of the synaptic device. The proposed framework was demonstrated for classification and regression tasks, using both off-chip and simplified on-chip sign-based learning techniques.

A Low-Energy High-Density Capacitor-Less I&F Neuron Circuit Using Feedback FET Co-Integrated With CMOS

We have developed a capacitor-less I&F neuron circuit with a dual gate positive feedback fieldeffect transistor (FBFET) and successfully co-integrated FBFET and CMOS in a wafer. By implementing the neuron circuit with FBFET, we can overcome the limits of conventional CMOS, reduce energy consumption, and imitate the biological neuron. The floating body of the FBFET can replace the membrane capacitor that occupies a large area and performs leaky integration of the neuron. Due to the extremely low sub-threshold swing of the FBFET (less than 0.528mv/dc), energy consumption of the neuron is significantly reduced by suppressing sub-threshold current. Finally, we analyzed the fabricated neuron circuit operation, retention time of the integrated charges and energy consumption compare to conventional CMOS neuron circuit.

Perspective: Uniform switching of artificial synapses for large-scale neuromorphic arrays

Resistive random-access memories are promising analog synaptic devices for efficient bio-inspired neuromorphic computing arrays. Here we first describe working principles for phase-change random-access memory, oxide random-access memory, and conductive-bridging random-access memory for artificial synapses. These devices could allow for dense and efficient storage of analog synapse connections between CMOS neuron circuits. We also discuss challenges and opportunities for analog synaptic devices toward the goal of realizing passive neuromorphic computing arrays. Finally, we focus on reducing spatial and temporal variations, which is critical to experimentally realize powerful and efficient neuromorphic computing systems.

Spiking Neural Networks with Unsupervised Learning Based on STDP Using Resistive Synaptic Devices and Analog CMOS Neuron Circuit.

We designed the CMOS analog integrate and fire (I&F) neuron circuit can drive resistive synaptic device. The neuron circuit consists of a current mirror for spatial integration, a capacitor for temporal integration, asymmetric negative and positive pulse generation part, a refractory part, and finally a back-propagation pulse generation part for learning of the synaptic devices. The resistive synaptic devices were fabricated using HfOx switching layer by atomic layer deposition (ALD). The resistive synaptic device had gradual set and reset characteristics and the conductance was adjusted by spike-timing-dependent-plasticity (STDP) learning rule. We carried out circuit simulation of synaptic device and CMOS neuron circuit. And we have developed an unsupervised spiking neural networks (SNNs) for 5 × 5 pattern recognition and classification using the neuron circuit and synaptic devices. The hardware-based SNNs can autonomously and efficiently control the weight updates of the synapses between neurons, without the aid of software calculations.

Learning in Memristor Crossbar-Based Spiking Neural Networks Through Modulation of Weight-Dependent Spike-Timing-Dependent Plasticity

In this paper, we propose a methodology to design learning systems based on a memristor crossbar structure. Learning is carried out with the help of a hardware-friendly spike-timing-dependent plasticity learning rule. Several simplifications and adaptations are made in order to apply the learning algorithm to memristor-based neural networks. The difficulties in conducting division and representing signed weights are circumvented using the proposed techniques. In addition, different conductance-changing behaviors are considered to demonstrate the effectiveness of the proposed algorithm and architecture when applied to practical memristor devices. Furthermore, various nonidealities existing in the neural network, including variations in CMOS neurons and memristor synapses and noises associated with updating the synaptic weights, are considered. We demonstrate that the proposed learning algorithm and hardware architecture are robust against most variations and noises. This robustness of learning is promising, as variations and stochastic behaviors of memristor devices are usually substantial. Memristor-based neural networks with the proposed learning algorithm are benchmarked with the MNIST handwritten digits recognition task. A recognition accuracy as high as 97.10% is demonstrated.

Dendritic-Inspired Processing Enables Bio-Plausible STDP in Compound Binary Synapses

Brain-inspired learning mechanisms, e.g. spike timing dependent plasticity (STDP), enable agile and fast on-the-fly adaptation capability in a spiking neural network. When incorporating emerging nanoscale resistive non-volatile memory (NVM) devices, with ultra-low power consumption and high-density integration capability, a spiking neural network hardware would result in several orders of magnitude reduction in energy consumption at a very small form factor and potentially herald autonomous learning machines. However, actual memory devices have shown to be intrinsically binary with stochastic switching, and thus impede the realization of ideal STDP with continuous analog values. In this work, a dendritic-inspired processing architecture is proposed in addition to novel CMOS neuron circuits. The utilization of spike attenuations and delays transforms the traditionally undesired stochastic behavior of binary NVMs into a useful leverage that enables biologically-plausible STDP learning. As a result, this work paves a pathway to adopt practical binary emerging NVM devices in brain-inspired neuromorphic computing.

Multiscaled Simulation Methodology for Neuro-Inspired Circuits Demonstrated with an Organic Memristor

Organic memristors are promising molecular electronic devices for neuro-inspired on-chip learning applications. In this paper, we present a numerically efficient compact model suitable for $Fe(bpy)_3^{2+}$ organic memristors operating according to an intramolecular charge transfer switching mechanism. This compact model, being physics-based and relying on electrical characterizations and parametric extractions performed on test structures, is especially efficient in pulsed mode and describes the conductance variations for both SET and RESET regimes. Using this model, a dynamic multiscale simulation approach has been set-up to extend the model from individual devices to larger model systems that learn progressively through time. To verify the soundness and highlight emergent properties of the organic memristors, instances of the compact model have been simulated within a simple neuromorphic design that co-integrates with CMOS neurons. In addition, a larger supervised learning system using the new compact model is demonstrated. These successful tests suggest our model might be of interest to neuromorphic designers.

Monolithic 3D neuromorphic computing system with hybrid CMOS and memristor-based synapses and neurons

Because of fabrication compatibility to current semiconductor technology, three-dimensional integrated circuits (3D-ICs) offer promising near-term solutions for maintaining Moore’s Law. 3D-ICs proffer high system speeds, massively parallel processing, low power consumption, and their high densities result in small footprints. In this paper, a novel 3D neuromorphic IC architecture which combines monolithic 3D integration and a synaptic array based on vertical resistive random-access memory structure (V-RRAM) is proposed. To analyze the electrical characteristics of the proposed synaptic array, a concise equivalent circuit model of the system is developed, and analytical calculations for each parameter of the equivalent circuit are provided. Moreover, a novel signal intensity encoding neuron design that can directly convert analog signal into a spiking waveform sequence is proposed and analyzed. A feasible 3D neuromorphic computing architecture is demonstrated. Applying the monolithic 3D integration technology on neuromorphic computing system hardware implementation can reduce the power consumption by 50%, and shrink die areas by 35%.

NbOx based oscillation neuron for neuromorphic computing

In a neuromorphic computing system, the complex CMOS neuron circuits have been the bottleneck for efficient implementation of weighted sum operation. The phenomenon of metal-insulator-transition (MIT) in strongly correlated oxides, such as NbO2, has shown the oscillation behavior in recent experiments. In this work, we propose using a MIT device to function as a compact oscillation neuron, achieving the same functionality as the CMOS neuron but occupying a much smaller area. Pt/NbOx/Pt devices are fabricated, exhibiting the threshold switching I-V hysteresis. When the NbOx device is connected with an external resistor (i.e., the synapse), the neuron membrane voltage starts a self-oscillation. We experimentally demonstrate that the oscillation frequency is proportional to the conductance of the synapse, showing its feasibility for integrating the weighted sum current. The switching speed measurement indicates that the oscillation frequency could achieve >33 MHz if parasitic capacitance can be eliminated.