Neural Network Circuit Research Articles

Memristor-Based Neural Network Circuit of Operant Conditioning With Bridging and Conditional Reinforcement

Memristor-Based Neural Network Circuit of Operant Conditioning With Bridging and Conditional Reinforcement

Computational Investigation of the Potential and Limitations of Machine Learning with Neural Network Circuits Based on Synaptic Transistors.

Synaptic transistors have been proposed to implement neuron activation functions of neural networks (NNs). While promising to enable compact, fast, inexpensive, and energy-efficient dedicated NN circuits, they also have limitations compared to digital NNs (realized as codes for digital processors), including shape choices of the activation function using particular types of transistor implementation, and instabilities due to noise and other factors present in analog circuits. We present a computational study of the effects of these factors on NN performance and find that, while accuracy competitive with traditional NNs can be realized for many applications, there is high sensitivity to the instability in the shape of the activation function, suggesting that, when highly accurate NNs are required, high-precision circuitry should be developed beyond what has been reported for synaptic transistors to date.

Memristor-Based Neural Network Circuit of Associative Memory With Occasion Setting

Memristor-Based Neural Network Circuit of Associative Memory With Occasion Setting

Spatial evolution of the proton-coupled Mott transition in correlated oxides for neuromorphic computing.

The proton-electron coupling effect induces rich spectrums of electronic states in correlated oxides, opening tempting opportunities for exploring novel devices with multifunctions. Here, via modest Pt-aided hydrogen spillover at room temperature, amounts of protons are introduced into SmNiO3-based devices. In situ structural characterizations together with first-principles calculation reveal that the local Mott transition is reversibly driven by migration and redistribution of the predoped protons. The accompanying giant resistance change results in excellent memristive behaviors under ultralow electric fields. Hierarchical tree-like memory states, an instinct displayed in bio-synapses, are further realized in the devices by spatially varying the proton concentration with electric pulses, showing great promise in artificial neural networks for solving intricate problems. Our research demonstrates the direct and effective control of proton evolution using extremely low electric field, offering an alternative pathway for modifying the functionalities of correlated oxides and constructing low-power consumption intelligent devices and neural network circuits.

Harnessing nonlinear conductive characteristic of TiO2/HfO2 memristor crossbar for implementing parallel vector–matrix multiplication

Memristor crossbar arrays are expected to achieve highly energy-efficient neuromorphic computing via implementing parallel vector–matrix multiplication (VMM) in situ. The similarities between memristors and neural synapses offer opportunities for realizing hardware-based brain-inspired computing, such as spike neural networks. However, the nonlinear I–V characteristics of the memristors limit the implementation of parallel VMM on passive memristor crossbar arrays. In our work, we propose to utilize differential conductance as a synaptic weight to implement linear VMM operations on a passive memristor array in parallel. We fabricated a TiO2/HfO2 memristor crossbar array, in which differential-conductance-based synaptic weight exhibits plasticity, nonvolatility, multi-states, and tunable ON/OFF ratio. The noise-dependent accuracy performance of VMM operations based on the proposed approach was evaluated, offering an optimization guideline. Furthermore, we demonstrated a spike neural network circuit capable of processing small spiking signals through the differential-conductance-based synapses. The experimental results showcase effective space-coded and time-coded spike pattern recognition. Importantly, our work opens up new possibilities for the development of passive memristor arrays, leading to increased energy and area efficiency in brain-inspired chips.

Memristor-Based Conditioned Inhibition Neural Network Circuit With Blocking Generalization and Differentiation

Memristor-Based Conditioned Inhibition Neural Network Circuit With Blocking Generalization and Differentiation

Memristor-based circuit design of episodic memory neural network and its application in hurricane category prediction

Episodic memory, as a type of long-term memory (LTM), is used to learn and store the unique personal experience. Based on the episodic memory biological mechanism, this paper proposes a bionic episodic memory memristive neural network circuit. The proposed memristive neural network circuit includes a neocortical module, a parahippocampal module and a hippocampus module. The neocortical module with the two paths structure is used to receive the sensory signal, and is also used to separate and transmit the spatial information and the non-spatial information involved in the sensory signal. The parahippocampal module is composed of the parahippocampal cortex-MEA and the perirhinal cortex-LEA, which receives the two types of information from the neocortical module respectively. As the last module, the hippocampus module receives and integrates the output information of the parahippocampal module as well as generates the corresponding episodic memory. Meanwhile, the specific scenario information with the certain temporal signal from the generated episodic memory is also extracted by the hippocampus module. The simulation results in PSPICE show that the proposed memristive neural network circuit can generate the various episodic memories and extract the specific scenario information successfully. By configuring the memristor parameters, the proposed bionic episodic memory memristive neural network circuit can be applied to the hurricane category prediction, which verifies the feasibility of this work.

Bisection Neural Network Toward Reconfigurable Hardware Implementation.

A hardware-friendly bisection neural network (BNN) topology is proposed in this work for approximately implementing massive pieces of complex functions in arbitrary on-chip configurations. Instead of the conventional reconfigurable fully connected neural network (FC-NN) circuit topology, the proposed hardware-friendly topology performs NN behaviors in a bisection structure, in which each neuron includes two constant synapse connections for both inputs and outputs. Compared with the FC-NN one, the reconfiguration of the BNN circuit topology eliminates the remarkable amount of dummy synapse connections in hardware. As the main target application, this work aims at building a general-purpose BNN circuit topology that offers a great amount of NN regressions. To achieve this target, we prove that the NN behaviors of the FC-NN circuit topologies can be migrated to the BNN circuit topologies equivalently. We introduce two approaches including the refining training algorithm and the inverted-pyramidal strategy to further reduce the number of neurons and synapses. Finally, we conduct the inaccuracy tolerance analysis to suggest the guideline for ultra-efficient hardware implementations. Compared with the state-of-the-art FC-NN circuit topology-based TrueNorth baseline, the proposed design can achieve 17.8- 22.2× hardware reduction and less than 1% inaccuracy.

Memristive Neural Network Circuit of Operant Conditioning With Reward Delay and Variable Punishment Intensity

Memristive Neural Network Circuit of Operant Conditioning With Reward Delay and Variable Punishment Intensity

PMONN: an optical neural network for photonic integrated circuits based on micro-resonator.

We propose an improved optical neural network (ONN) circuit architecture based on conventional micro-resonator ONNs, called the Phase-based Micro-resonator Optical Neural Network (PMONN). PMONN's core architecture features a Convolutions and Batch Normalization (CB) unit, comprising a phase-based (PB) convolutional layer, a Depth-Point-Wise (DPW) convolutional layer, and a reconstructed Batch Normalization (RBN) layer. The PB convolution kernel uses modulable phase shifts of Add-drop MRRs as learnable parameters and their optical transfer function as convolution weights. The DPW convolution kernel amplifies PB convolution weights by learning the amplification factors. To address the internal covariate shift during training, the RBN layer normalizes DPW outputs by reconstructing the BN layer of the electronic neural network, which is then merged with the DPW layer in the test stage. We employ the tunable DAs in the architecture to implement the merged layer. PMONN achieves 99.15% and 91.83% accuracy on MNIST and Fashion-MNIST datasets, respectively. This work presents a method for implementing an optical neural network on the improved architecture based on MRRs and increases the flexibility and reusability of the architecture. PMONN has potential applications as the backbone for future optical object detection neural networks.

Memristive neural network circuit design based on locally competitive algorithm for sparse coding application

Sparse coding can quickly, accurately, and inexpensively represent the stimulus information received by biological vision neurons. However, there is no entire circuit that can realize real-time sparse coding by efficient analog computation. A memristor neural network circuit that solves the sparse coding problem in real-time and in parallel is proposed to solve such a problem. In our circuit, a novel memristor array structure can realize both reading and writing in parallel. A neural network circuit that can execute the Locally Competitive Algorithm (LCA) is designed based on this structure. Given these designs, the proposed neural networks circuit can utilize the programmability of the memristor array to real-time process various sparse coding problems. Based on the proposed circuit, the module can process binary and grayscale image sparse coding, providing a circuit implementation platform for sparse coding tasks. The simulation results demonstrate that the processing speed of sparse coding is improved compared with the MATLAB simulation and the application robustness is enhanced.

Mosaic: in-memory computing and routing for small-world spike-based neuromorphic systems

The brain’s connectivity is locally dense and globally sparse, forming a small-world graph—a principle prevalent in the evolution of various species, suggesting a universal solution for efficient information routing. However, current artificial neural network circuit architectures do not fully embrace small-world neural network models. Here, we present the neuromorphic Mosaic: a non-von Neumann systolic architecture employing distributed memristors for in-memory computing and in-memory routing, efficiently implementing small-world graph topologies for Spiking Neural Networks (SNNs). We’ve designed, fabricated, and experimentally demonstrated the Mosaic’s building blocks, using integrated memristors with 130 nm CMOS technology. We show that thanks to enforcing locality in the connectivity, routing efficiency of Mosaic is at least one order of magnitude higher than other SNN hardware platforms. This is while Mosaic achieves a competitive accuracy in a variety of edge benchmarks. Mosaic offers a scalable approach for edge systems based on distributed spike-based computing and in-memory routing.

A Memristive Spiking Neural Network Circuit for Bio-inspired Navigation Based on Spatial Cognitive Mechanisms.

Cognitive navigation, a high-level and crucial function for organisms' survival in nature, enables autonomous exploration and navigation within the environment. However, most existing works for bio-inspired navigation are implemented with non-neuromorphic computing. This work proposes a bio-inspired memristive spiking neural network (SNN) circuit for goal-oriented navigation, capable of online decision-making through reward-based learning. The circuit comprises three primary modules. The place cell module encodes the agent's spatial position in real-time through Poisson spiking; the action cell module determines the direction of subsequent movement; and the reward-based learning module provides a bio-inspired learning method adaptive to delayed and sparse rewards. To facilitate practical application, the entire SNN is quantized and deployed on a real memristive hardware platform, achieving about a 21× reduction in energy consumption compared to a typical digital acceleration system in the forward computing phase. This work offers an implementation idea of neuromorphic solution for robotic navigation application in low-power scenarios.

Neural Network Circuits for Bionic Associative Memory and Temporal Order Memory Based on DNA Strand Displacement.

Pavlovian associative memory plays an important role in our daily life and work. The realization of Pavlovian associative memory at the deoxyribonucleic acid (DNA) molecular level will promote the development of biological computing and broaden the application scenarios of neural networks. In this article, bionic associative memory and temporal order memory circuits are constructed by DNA strand displacement (DSD) reactions. First, a temporal logic gate is constructed on the basis of DSD circuit and extended to a three-input temporal logic gate. The output of temporal logic gate is used for the weight species of associative memory. Second, the forgetting module and output module based on the DSD circuit are constructed to realize some functions of associative memory, including associative memory with simultaneous stimulus, associative memory with interstimulus interval effect, and the facilitation by intermittent stimulus. In addition, the coding, storage, and retrieval modules are designed based on the analysis and memory capabilities of temporal logic gate for temporal information. The temporal order memory circuit is constructed, demonstrating the temporal order memory ability of DNA circuit. Finally, the reliability of the circuit is verified through Visual DSD software simulation. Our work provides ideas and inspiration to construct more complex DNA bionic circuits and intelligent circuits by using DSD technology.

Memristor-Based Neural Network Circuit of Full-Function Pavlov Associative Memory With Unconditioned Response Mechanisms

Memristor-Based Neural Network Circuit of Full-Function Pavlov Associative Memory With Unconditioned Response Mechanisms

Memristor-Based Neural Network Circuit of Associative Memory With Overshadowing and Emotion Congruent Effect.

Most memristor-based neural network circuits consider only a single pattern of overshadowing or emotion, but the relationship between overshadowing and emotion is ignored. In this article, a memristor-based neural network circuit of associative memory with overshadowing and emotion congruent effect is designed, and overshadowing under multiple emotions is taken into account. The designed circuit mainly consists of an emotion module, a memory module, an inhibition module, and a feedback module. The generation and recovery of different emotions are realized by the emotion module. The functions of overshadowing under different emotions and recovery from overshadowing are achieved by the inhibition module and the memory module. Finally, the blocking caused by long-term overshadowing is implemented by the feedback module. The proposed circuit can be applied to bionic emotional robots and offers some references for brain-like systems.

A Memristor-Based Neural Network Circuit With Latent Inhibition and Transient Forgetting Effects and Application in Industrial Intelligent Grasping

A Memristor-Based Neural Network Circuit With Latent Inhibition and Transient Forgetting Effects and Application in Industrial Intelligent Grasping

Elman neural network and Thevenin equivalent circuit model based multi-measurement Kalman filter for SOC estimation

Elman neural network and Thevenin equivalent circuit model based multi-measurement Kalman filter for SOC estimation

Memristor-Based Neural Network Circuit of Duple-Reward and Duple-Punishment Operant Conditioning With Time Delay

Memristor-Based Neural Network Circuit of Duple-Reward and Duple-Punishment Operant Conditioning With Time Delay

Memristor-Based Neural Network Circuit of Emotional Habituation With Contextual Dependency

Most memristor-based neural networks only consider habituation under repeated stimuli, but the emotional habituation under repeated emotional stimuli is ignored. In this paper, a memristor-based neural network circuit that can realize emotional habituation with contextual dependency is proposed. The designed circuit consists of habituation module, emotion control module, context control module and generalization module. When different emotional stimuli are applied, the emotion control module produces different feedback voltages and has an impact on the formation of habituation. In addition, the influence of contextual information on habituation is considered and the contextual dependency of habituation is achieved. Finally, the generalization of habituation is achieved through the generalization module. The simulation results in PSPICE indicate that the proposed circuit can realize emotional habituation with contextual dependency. The neural network circuit of emotional habituation with contextual dependency provides some references for brain-like intelligence.