Neural Network Circuit Research Articles

Periodicity and multi-periodicity generated by impulses control in delayed Cohen–Grossberg-type neural networks with discontinuous activations

This paper discusses the periodicity and multi-periodicity in delayed Cohen–Grossberg-type neural networks (CGNNs) possessing impulsive effects, whose activation functions possess discontinuities and are allowed to be unbounded or nonmonotonic. Based on differential inclusion and cone expansion–compression fixed-point theory of set-valued mapping, several improved criteria are given to derive the positive solution with ω-periodicity and ω-multi-periodicity for delayed CGNNs under impulsive control. These ω-periodicity/ω-multi-periodicity orbits are produced by impulses control. The analytical method and theoretical results presented in this paper are of certain significance to the design of neural network models or circuits possessing discontinuous neuron activation and impulsive effects in periodic environment.

Competitive Neural Network Circuit Based on Winner-Take-All Mechanism and Online Hebbian Learning Rule

In this article, we design a memristive competitive neural network circuit based on the winner-take-all (WTA) mechanism and the online Hebbian learning rule. Each synapse of the network contains two memristors whose terminals of signal inputs are opposite. However, only one memristor participates in the calculation each time, and that one is determined by the original input signal. The competitive neural network circuit includes two parts: forward calculation and weight update. In this article, the forward calculation part of the circuit is designed based on the WTA mechanism. The combination of the leaky-integrate-and-fire (LIF) model and pMOS realizes the lateral inhibition of neurons. The design of the weight updating part is based on Hebbian learning rules. In each cycle, only synaptic memristors connected to the winner output neuron in forward calculation can be adjusted. The voltage used for synaptic memristor adjustment comes from the membrane voltage of the winner output neuron. The whole neural network circuit does not need the participation of a central processing unit (CPU) or a field-programmable gate array (FPGA) and really realizes parallel calculation, the saving of area, power consumption, and a certain extent computing-in-memory. Based on the circuit designed in PSPICE, we simulated the classification of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5\times3$ </tex-math></inline-formula> pixel pictures. The changing trend of weights in the training phase and the high recognition accuracy in the recognition phase prove that the network can learn and recognize different patterns. The competitive neural network can be applied to the neuromorphic system of visual pattern recognition.

Memristor-Based Neural Network Circuit of Emotion Congruent Memory With Mental Fatigue and Emotion Inhibition.

Most memristor-based neural networks only consider a single mode of memory or emotion, but ignore the relationship between emotion and memory. In this paper, a memristor-based neural network circuit of emotion congruent memory is proposed and verified by the simulation results. The designed circuit consists of a memory module, an emotion module and an association neuron module. Varieties of memory and emotion functions are considered. The functions such as learning, forgetting, variable rate and emotion generation are implemented by the circuit. Furthermore, mental fatigue and emotion inhibition which are two important self-protective measures of the brain are realized in this paper on the basis of emotion congruent memory. Finally, the paper also considers the congruence between emotion and memory materials and the regulation of emotion on memory. The neural network circuit of emotion congruent memory can provide more references for the application of memristor.

Basin of Attraction Analysis of New Memristor-Based Fractional-Order Chaotic System

Memristor is the fourth basic electronic element discovered in addition to resistor, capacitor, and inductor. It is a nonlinear gadget with memory features which can be used for realizing chaotic, memory, neural network, and other similar circuits and systems. In this paper, a novel memristor-based fractional-order chaotic system is presented, and this chaotic system is taken as an example to analyze its dynamic characteristics. First, we used Adomian algorithm to solve the proposed fractional-order chaotic system and yield a chaotic phase diagram. Then, we examined the Lyapunov exponent spectrum, bifurcation, SE complexity, and basin of attraction of this system. We used the resulting Lyapunov exponent to describe the state of the basin of attraction of this fractional-order chaotic system. As the local minimum point of Lyapunov exponential function is the stable point in phase space, when this stable point in phase space comes into the lowest region of the basin of attraction, the solution of the chaotic system is yielded. In the analysis, we yielded the solution of the system equation with the same method used to solve the local minimum of Lyapunov exponential function. Our system analysis also revealed the multistability of this system.

Memristor-based Hopfield network circuit for recognition and sequencing application

Hopfield neural network has been widely used in image recognition because of its associative memory behavior. In this paper, a memristor neural network circuit is designed, which can recognize and sequence four characters simultaneously. It mainly includes three modules, namely a character recognition module, a signal processing module and a sequence module. The character recognition module consists of four individual character recognition units, corresponding to the recognition of four character images (W, H, A, T). The character recognition module includes calculation submodule and iteration submodule. After the operation of the calculation submodule and the iterative submodule, the four-character images distributed by noise can be identified simultaneously. The signal processing module is used to simplify the output signals of the character recognition module by four adder units. The sequence module ensures that stable state is eventually converged to the word (WHAT). The synapse weight circuit given in this paper can obtain different weights, so as to realize the function of associative memory. The iterative process circuit of Hopfield neural network is also designed to further demonstrate the iterative process. The neural network circuit composed of memristors maybe smaller, which may provide a reference for the development of new brain-like system.

Design of a single-electron neural network circuit controlling weights for reservoir computing

We propose a weight function controllable circuit for single-electron (SE) reservoir computing (RC). While an SE circuit has advantages, e.g., low power consumption and non-linear operation, it also has the disadvantage of being sensitive to fluctuation. Therefore, we focus on RC, as weights do not need to be adjusted in the reservoir layer and there is redundancy for noise. Our idea is that introducing RC can overcome the disadvantages of an SE circuit. Our designed circuit expresses and adjusts the weights by determining how many element signals can pass through it. The result of an operation test indicates that our circuit could accurately express the weights and change them correctly. We also evaluated the discrimination of input signals and found that our circuit could accurately do this. Therefore, our SE RC circuit has the potential to become a new information processing circuit that overcomes the problems of the SE circuit.

Stable and compact design of Memristive GoogLeNet Neural Network

According to the requirements of edge intelligence for circuit volume, power consumption and computing performance, a Memristive GoogLeNet Neural Network (MGNN) circuit is designed using memristor which is a new device integrating storage and computing as the basic circuit element. This circuit adopts 1×1 convolution and multi-scale convolution feature fusion to reduce the number of layers required by the network while ensuring the recognition accuracy of circuit. In order to reduce the size of the memristor crossbars in the circuit, we design word-line pruning and bit-line pruning methods of Memristive Convolution (MC) layers. We also use the parameter distribution of the memristive neural network to further reduce the size of memristor crossbars. The Memristive Batch Normalization (MBN) layer and Memristive Dropout (MD) are merged into front MC layers according mathematical analysis for cutting the number of network layers and decreasing the power consumption of the circuit. We also design the channel optimization and layer optimization methods of MC layers which greatly reduce the negative effect of multi-state conductance of memristors on the accuracy, improve the stability of the circuit, and reduce the circuit volume and power consumption. Experiments show that this circuit can get 89.83% accuracy on the CIFAR-10 data set, and the power consumption of a single neuron is only 1.3μW. When the number of memristor multi-state conductance is 24=16, the accuracy of the MGNN circuit close to float MGNN can still be obtained.

Cryptosystem for Grid Data Based on Quantum Convolutional Neural Networks and Quantum Chaotic Map

Motivated by the existing circuit model of quantum convolutional neural network, a new quantum convolutional neural network circuit model is devised, which is combined with quantum chaotic map to construct a symmetric cryptosystem. Quantum chaotic map produces key stream for encryption and decryption. The cryptosystem simulates the basic process of communication. Theoretical analysis manifests that the cryptosystem is effective. Additionally, simulation experiments based on MNIST data set show that the cryptosystem is secure. Furthermore, the proposed cryptosystem can be applied not only for image data, but for text data. Therefore, the grid data can be encrypted by utilizing the cryptosystem.

A co-design framework of neural networks and quantum circuits towards quantum advantage

Despite the pursuit of quantum advantages in various applications, the power of quantum computers in executing neural network has mostly remained unknown, primarily due to a missing tool that effectively designs a neural network suitable for quantum circuit. Here, we present a neural network and quantum circuit co-design framework, namely QuantumFlow, to address the issue. In QuantumFlow, we represent data as unitary matrices to exploit quantum power by encoding n = 2k inputs into k qubits and representing data as random variables to seamlessly connect layers without measurement. Coupled with a novel algorithm, the cost complexity of the unitary matrices-based neural computation can be reduced from O(n) in classical computing to O(polylog(n)) in quantum computing. Results show that on MNIST dataset, QuantumFlow can achieve an accuracy of 94.09% with a cost reduction of 10.85 × against the classical computer. All these results demonstrate the potential for QuantumFlow to achieve the quantum advantage.

Memristor-based BAM circuit implementation for image associative memory and filling-in

Associative memory and filling-in are two essential functions of the human brain. To implement these two brain-inspired functions in hardware, we proposed a memristor-based bidirectional associative memory (BAM) circuit in this paper. This circuit combines an online algorithm with a memristor array adjustment process, thus makes the circuit more universal for various tasks. The proposed circuit is constructed out of memristive synaptic circuits, IN modules and ACT modules. The memristive synaptic circuits utilize memristor arrays to represent weight matrix and operate corresponding operations hence make computing-in-memory and process information in parallel, which simplifies the complexity of circuit and improves the processing speed. The IN modules employ transistors as switches to choose the input layer hence can get initial information flow bidirectionally. The ACT modules perform activation function and can output continuous arbitrary real numbers. Thereby, both binary and gray-scale images can be tested in the proposed circuit. In addition to the hetero-association and filling-in results given in detail, the retrieval rates of the proposed circuit with the impact of different degrees of noise and the number of stored patterns are also evaluated and compared with software-based BAM. The simulation has experimented via MATLAB and PSpice, and the corresponding results show a remarkable performance of the proposed circuit. The influence of memristor’s stuck-at-fault is also considered. In comparison with software-based BAM and similar memristor-based neural network circuit, the proposed circuit performs better in processing speed.

An Efficient Memristor-Based Circuit Implementation of Squeeze-and-Excitation Fully Convolutional Neural Networks.

Recently, there has been a surge of interest in applying memristors to hardware implementations of deep neural networks due to various desirable properties of the memristor, such as nonvolativity, multivalue, and nanosize. Most existing neural network circuit designs, however, are based on generic frameworks that are not optimized for memristors. Furthermore, to the best of our knowledge, there are no existing efficient memristor-based implementations of complex neural network operators, such as deconvolutions and squeeze-and-excitation (SE) blocks, which are critical for achieving high accuracy in common medical image analysis applications, such as semantic segmentation. This article proposes convolution-kernel first (CKF), an efficient scheme for designing memristor-based fully convolutional neural networks (FCNs). Compared with existing neural network circuits, CKF enables effective parameter pruning, which significantly reduces circuit power consumption. Furthermore, CKF includes the novel, memristor-optimized implementations of deconvolution layers and SE blocks. Simulation results on real medical image segmentation tasks confirm that CKF obtains up to 56.2% reduction in terms of computations and 33.62-W reduction in terms of power consumption in the circuit after weight pruning while retaining high accuracy on the test set. Moreover, the pruning results can be applied directly to existing circuits without any modification for the corresponding system.

Memristor-Based Neural Network Circuit of Delay and Simultaneous Conditioning

Most conventional memristor-based Pavlov associative memory neural network circuits have been working on realizing the learning and forgetting functions of simultaneous conditioning. However, the time interval between unconditional stimulus and conditional stimulus is a critical variable in classical conditioning. Different unconditional and conditional stimulus intervals evoke associative memory of brains with different rates. For example, learning in simultaneous conditioning is less effective than delay conditioning. Therefore, a memristor-based neural network circuit of delay and simultaneous conditioning is designed. The proposed circuit consists of learning states detection module, voltage control module, and synapse module. Many functions, such as short-delay conditioning learning, long-delay conditioning learning, simultaneous conditioning learning, experience learning, and two types of forgetting are implemented by the circuit. In particular, the so-called experience learning is that learning the forgotten knowledge will be faster than before and learning again will forget more slowly. This function is more bionic. The correctness of the design is demonstrated through simulations using PSPICE.

Cyclic reservoir neural network circuit for 3D IC implementation

Cyclic reservoir neural network circuit for 3D IC implementation

Memristor-based neural network circuit of pavlov associative memory with dual mode switching

There are many learning modes in associative memory, but most of memristor-based Pavlov associative memory circuits only have a single mode. A learning circuit that can realize Pavlov associative memory with dual-mode switching is designed and verified by the simulation results. The designed circuit consists of the auditory (visual) synapse, auditory (visual) control voltage module and auditory (visual) inhibition module. This paper considers two different learning modes, auditory mode and visual mode. The modes can run not only separately but also alternately. Besides, the paper also considers the inhibition relationship between learning modes and reflects the relationship by adjusting the influence of mode switching period on the learning speed of modes. As a widespread phenomenon in the natural world, mode inhibition of associative memory can be regarded as a psychological process of living things under different external stimulus. Mode inhibition is an interesting subject, which can allow the artificial neural network to mimic more realistic situations of memory. The dual mode switching is an essential part of life, which can provide more references for the practical application of memristor.

Concept of LIF Neuron Circuit for Rate Coding in Spike Neural Networks

The study presents an oscillator circuit for a spike neural network with the possibility of firing rate coding and sigmoid-like activation function. The circuit contains a switching element with an S-shaped current-voltage characteristic and two capacitors; one of the capacitors is shunted by a control resistor. The circuit is characterised by a strong dependence of the frequency of relaxation oscillations on the magnitude of the control resistor. The dependence has a sigmoid-like form and we present an analytical method for dependence calculation. Finally, we describe the concept of the spike neural network architecture with firing rate coding based on the presented circuit for creating neuromorphic devices and artificial intelligence.

Analog Self-Timed Programming Circuits for Aging Memristors

Reliable programming crossbar memristors to the required resistive states is the challenge that hinders VLSI deployment of the memristive neural network circuits, as current memristive devices face the variability issues of resistive switching. There is also a need for on-chip control circuitry that detects malfunctioning memristive nodes in the crossbar due to the memristor aging. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Program and Verify</i> ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$P\&amp;V$ </tex-math></inline-formula> ) schemes can be used for both controlling resistive switching as well as evaluating the functionality of memristors. In this brief, we propose a novel analog circuit design for the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$P\&amp;V$ </tex-math></inline-formula> approach of row-by-row programming bipolar memristors in a 1T1M crossbar configuration. The proposed control circuit (CC) is self-timed and performs both read and program operations, decreasing the overall programming complexity. CC design is verified with Spice simulations using low power 22nm high-k CMOS models and Modified S memristor model for large scale simulations. Parasitic of wire lines under thermal variation and CMOS variability were included for programming 1T1M crossbar partitions of the sizes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$16\times 16$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$32\times 32$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$64\times 64$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$128\times 128$ </tex-math></inline-formula> .

A pathoconnectome of early neurodegeneration: Network changes in retinal degeneration

Connectomics has demonstrated that synaptic networks and their topologies are precise and directly correlate with physiology and behavior. The next extension of connectomics is pathoconnectomics: to map neural network synaptology and circuit topologies corrupted by neurological disease in order to identify robust targets for therapeutics. In this report, we characterize a pathoconnectome of early retinal degeneration. This pathoconnectome was generated using serial section transmission electron microscopy to achieve an ultrastructural connectome with 2.18nm/px resolution for accurate identification of all chemical and gap junctional synapses. We observe aberrant connectivity in the rod-network pathway and novel synaptic connections deriving from neurite sprouting. These observations reveal principles of neuron responses to the loss of network components and can be extended to other neurodegenerative diseases.

A Generic Nano-Watt Power Fully Tunable 1-D Gaussian Kernel Circuit for Artificial Neural Network

This brief presents an ultra-low-power generic fully tunable analog 1-D Gaussian kernel (GK) circuit, which is employed as an activation neuron for the radial basis function artificial neural network. In the proposed GK circuit, the maximum likelihood, center, and width of the Gaussian profile can be independently controlled. Besides, we have developed a mathematical model for the proposed GK circuit and further verified them experimentally. Thereby, the presented modeling can be employed to facilitate the off-chip learning of the neural network hardware. Fabricated in 180 nm CMOS process, the prototype demonstrates the full tunability of the proposed GK circuit and good agreement between the experimental measurements and mathematical model. The relative error of the measured width of the GK’s profile is less than 7 % compared to the value predicted by the presented mathematical model. The total power consumption is 13.5 nW with a supply voltage of 0.9 V, and the core circuit occupies 0.013 mm2.

Fixed-Time Passification Analysis of Interconnected Memristive Reaction-Diffusion Neural Networks

This article deals with fixed-time passification problem of interconnected networks composed of multiple memristive reaction-diffusion subsystems. Different from the finite-time passivity proposed by Wang, Zhang, et al. (2018), a novel concept of fixed-time passivity/passification is proposed by using upper right Dini derivative, where the settling time is independent on initial value. Next, by designing an appropriate controller and utilizing inequality technique, we put forward several sufficient conditions to guarantee the fixed-time passification of the interconnected memristive reaction-diffusion neural networks (MRDNNs). Furthermore, a fixed-time synchronization criterion is proposed for interconnected MRDNNs. The fixed-time passivity/passification of interconnected MRDNNs has important practical significance in the design and implementation of neural network circuits. Finally, a numerical example is presented to substantiate the correctness of the theoretical results.

Adaptive Control Using NARMA-L2 Model and Electric Circuit for Condenser of Thermal Power Plant

Design and implantation electric circuit for enhancement performance of steam power plant and artificial neural networks technique used to control of condenser. Artificial neural network has been applied to control of the important variables of condenser in Al–Dura power plant in Baghdad such as pressure, temperature. in this study, applied Simulink model in Matlab program (v 2014 a) by using artificial neural network toolbox. the model of condenser of neural network by using NARMA to generated data and train network and using back propagation algorithm for training neural network. The method of control by ANN is off line. the electric circuit of pressure sensor and temperature sensor consist of transformer, dc bridge and voltage regulator. Experimental data of actual power plant obtained from al-dura power plant. comparing results of modelling neural network and electric circuit with experimental data of actual power plant. the results shown the maximum deviation between the ANN and electric circuit with experimental data is less than 4%. artificial neural network can be used in many industrial and engineering application.