Memristive Networks Research Articles

Asymmetrical Training Scheme of Binary-Memristor-Crossbar-Based Neural Networks for Energy-Efficient Edge-Computing Nanoscale Systems.

For realizing neural networks with binary memristor crossbars, memristors should be programmed by high-resistance state (HRS) and low-resistance state (LRS), according to the training algorithms like backpropagation. Unfortunately, it takes a very long time and consumes a large amount of power in training the memristor crossbar, because the program-verify scheme of memristor-programming is based on the incremental programming pulses, where many programming and verifying pulses are repeated until the target conductance. Thus, this reduces the programming time and power is very essential for energy-efficient and fast training of memristor networks. In this paper, we compared four different programming schemes, which are F-F, C-F, F-C, and C-C, respectively. C-C means both HRS and LRS are coarse-programmed. C-F has the coarse-programmed HRS and fine LRS, respectively. F-C is vice versa of C-F. In F-F, both HRS and LRS are fine-programmed. Comparing the error-energy products among the four schemes, C-F shows the minimum error with the minimum energy consumption. The asymmetrical coarse HRS and fine LRS can reduce the time and energy during the crossbar training significantly, because only LRS is fine-programmed. Moreover, the asymmetrical C-F can maintain the network’s error as small as F-F, which is due to the coarse-programmed HRS that slightly degrades the error.

Study of Recall Time of Associative Memory in a Memristive Hopfield Neural Network

By associative memory, people can remember a pattern in microseconds to seconds. In order to emulate human memory, an artificial neural network should also spend a reasonable time in recalling matters of different task difficulties or task familiarities. In this paper, we study the recall time in a memristive Hopfield network (MHN) implemented with memristor-based synapses. With the operating frequencies of 1-100 kHz, patterns can be stored into the network by altering the resistance of the memristors, and the pre-stored patterns can be successfully recalled, being similar to the associative memory behavior. For the same target pattern (the same familiarity), recall time of the MHN varies with the inputs, which is similar to the effect in the human brain that recall time depends on task difficulty. On the other hand, for the same input (i.e., the same initial state), the recall time may be different for different target patterns, which is similar to the effect in the brain that recall time depends on the familiarity. In addition, the effect of stimulation (updating frequency) on recall time may be complicated: a higher stimulation frequency may not always lead to a faster recall (it may even slow the recalling process in some circumstances). Our memristive Hopfield network shows good potential in mimicking the characteristics of human associative memory.

Dynamical attractors of memristors and their networks

It is shown that the time-averaged dynamics of memristors and their networks periodically driven by alternating-polarity pulses may converge to fixed-point attractors. Starting with a general memristive system model, we derive basic equations describing the fixed-point attractors and investigate attractors in the dynamics of ideal, threshold-type and second-order memristors, and memristive networks. A memristor potential function is introduced, and it is shown that in some cases the attractor identification problem can be mapped to the problem of potential function minimization. Importantly, the fixed-point attractors may only exist if the function describing the internal state dynamics depends on an internal state variable. Our findings may be used to tune the states of analog memristors, and also be relevant to memristive synapses subjected to forward- and back-propagating spikes.

Memristive networks: From graph theory to statistical physics

We provide an introduction to a very specific toy model of memristive networks, for which an exact differential equation for the internal memory which contains the Kirchhoff laws is known. In particular, we highlight how the circuit topology enters the dynamics via an analysis of directed graph. We try to highlight in particular the connection between the asymptotic states of memristors and the Ising model, and the relation to the dynamics and statics of disordered systems.

Memristive LSTM Network for Sentiment Analysis

This paper presents a complete solution for the hardware design of a memristor-based long short-term memory (MLSTM) network. Throughout the design process, we fully consider the external and internal structures of the long short-term memory (LSTM), both of which are efficiently implemented by memristor crossbars. In the specific design of the internal structure, the parameter sharing mechanism is used between the LSTM cells to minimize the hardware design scale. In particular, we designed a circuit that requires only one memristor crossbar for each unit in the LSTM cell. The activation function, including sigmoid and tanh (hyperbolic tangent function), involved in each unit is approximated by a piecewise function, which is designed with the corresponding hardware. To verify the effectiveness of the system we designed, we test it on IMDB and SemEval datasets. Considering the huge impact of the dimensions of the input data on the scale of the hardware design, we use word2vector instead of one-hot encoding for the input data encoding. With the parameter sharing mechanism, the transformed vectors are input in different periods, so only 65 memristive crossbars are needed in the entire system to complete the sentiment analysis of the input text. The experimental results verify the effectiveness of our proposed MLSTM system.

Analysis and Verification of a Novel Single-Photon Memristor

This work reports a device based on single-photon memristor. In this paper, the equivalent circuit model of single-photon memristor was established, and we stimulated three distinctive signatures of general memristors by this model. Through the incident photon is absorbed, the device goes from the avalanche state to the quenching state, then back to the equilibrium state, and we obtained the experimental results. The results of three distinctive signatures of general memristors are verified by the test of the real single-photon memristor, which is fabricated with 0.18 μm CIS technology. On the basis of the memristive characteristics of this device, it can be applied in chaotic circuit integrated with single-photon memristor and neural network.

Neuromorphic Vision Hybrid RRAM-CMOS Architecture

The development of a bioinspired image sensor, which can match the functionality of the vertebrate retina, has provided new opportunities for vision systems and processing through the realization of new architectures. Research in both retinal cellular systems and nanodriven memristive technology has made a challenging arena more accessible to emulate features of the retina that are closer to biological systems. This paper synthesizes the signal flow path of photocurrent throughout a retina in a scalable 180-nm CMOS technology, which initiates at a $128\times 128$ active pixel image sensor, and converges to a $16\times 16$ array, where each node emits a spike train synonymous to the function of the retinal ganglionic output cell. This signal can be sent to the visual cortex for image interpretation as part of an artificial vision system. Layers of memristive networks are used to emulate the functions of horizontal and amacrine cells in the retina, which average and converge signals. The resulting image matches biologically verified results within an error margin of 6% and exhibits the following features of the retina: lateral inhibition, asynchronous adaptation, and a low-dynamic-range integration active pixel sensor to perceive a high-dynamic-range scene.

Input-to-state stability of impulsive inertial memristive neural networks with time-varying delayed

The property of input-to-state stability (ISS) of inertial memristor-based neural networks with impulsive effects is studied. Firstly, according to the characteristics of memristor and inertial neural networks, the inertial memristor-based neural networks are built. Secondly, based on the impulsive control theory, the average impulsive interval approach, Halanay differential inequality, Lyapunov method and comparison property, some sufficient conditions ensuring ISS of the inertial memristor-based neural networks under impulsive controller are derived. In this paper, we consider two types of impulse, stabilizing impulses and destabilizing impulses. When the inertial memristor-based neural networks are originally not ISS, by choosing a suitable lower bound of the average impulsive interval, the stabilizing impulses can be used to stabilize the inertial memristor-based neural networks. On the contrary, the inertial memristor-based neural networks are originally ISS, by restricting the upper bound of the average impulsive interval, the ISS of inertial memristor-based neural networks with destabilizing impulses can be ensured. Finally, numerical results are presented to illustrate the main results.

Memristive retinomorphic grid architecture removing noise and preserving edge

In this paper, special memristor circuit and memristive retina network structure for analogue image processing have been presented. The new developments on memristor element have opened various possibilities due to its being in nano-scale and having nonlinear behavior. The proposed memristor emulator consists of only one Operational Transconductance Amplifier (OTA) and two MOS transistors which are operated in sub-threshold region. The memristor fuse structure is obtained by connecting two proposed memristor emulators. In the second section of our paper, we proposed a circuit block which is composed of 16 × 16 pixels retinomorphic memristive grid to maintain a smoothed and edges preserved image. All simulation results for both proposed memristor circuits and retinomorphic grid are obtained as expected.

Memristive Fully Convolutional Network: An Accurate Hardware Image-Segmentor in Deep Learning

As well known, fully convolutional network (FCN) becomes the state of the art for semantic segmentation in deep learning. Currently, new hardware designs for deep learning have focused on improving the speed and parallelism of processing units. This motivates memristive solutions, in which the memory units (i.e., memristors) have computing capabilities. However, designing a memristive deep learning network is challenging, since memristors work very differently from the traditional CMOS hardware. This paper proposes a complete solution to implement memristive FCN (MFCN). Voltage selectors are firstly utilized to realize max-pooling layers with the detailed MFCN deconvolution hardware circuit by the massively parallel structure, which is effective since the deconvolution kernel and the input feature are similar in size. Then, deconvolution calculation is realized by converting the image into a column matrix and converting the deconvolution kernel into a sparse matrix. Meanwhile, the convolution realization in MFCN is also studied with the traditional sliding window method rather than the large matrix theory to overcome the shortcoming of low efficiency. Moreover, the conductance values of memristors are predetermined in Tensorflow with ex-situ training method. In other words, we train MFCN in software, then download the trained parameters to the simulink system by writing memristor. The effectiveness of the designed MFCN scheme is verified with improved accuracy over some existing machine learning methods. The proposed scheme is also adapt to LFW dataset with three-classification tasks. However, the MFCN training is time consuming as the computational burden is heavy with thousands of weight parameters with just six layers. In future, it is necessary to sparsify the weight parameters and layers of the MFCN network to speed up computing.

Learning mechanisms in memristor networks based on GaN nanomembranes

We demonstrate experimentally that single crystalline GaN nanomembranes arranged in simple networks exhibit learning mechanisms such as habituation and dishabituation followed by storage of the response to a certain electrical stimulus. These artificial learning mechanisms are analogous to non-associative learning processes which are identical in simple animals and human beings. We found that the learning time depends on the number of GaN membranes in parallel, and this parameter decreases by 30% when three memristors are connected in parallel compared to the learning time of a single memristor. Moreover, an increased number of parallel memristors reduces the eventual asymmetry in the temporal response of the circuit at positive and negative step voltages.

Image Edge Detection Based on Swarm Intelligence Using Memristive Networks

Recent advancements in the development of memristive devices has opened new opportunities for hardware implementation of new computing models. Researchers have shown the suitability of memristive devices for swarm intelligence algorithms to solve a maze in hardware. In this paper, we utilize swarm intelligence of memristive networks to perform image edge detection. First, we propose a hardware-friendly algorithm for image edge detection based on ant colony optimization. Second, we implement the image edge detection algorithm using memristive networks. Furthermore, we explain the impact of various parameters of the memristors on the efficacy of the implementation. Our results show 28% improvement in the energy compared to a low power CMOS hardware implementation based on stochastic circuits. Furthermore, our design occupies up to $5\boldsymbol \times $ less area.

Evaluation of the computational capabilities of a memristive random network (MN3) under the context of reservoir computing

This work presents the simulation results of a novel recurrent, memristive neuromorphic architecture, the MN3 and explores its computational capabilities in the performance of a temporal pattern recognition task by considering the principles of the reservoir computing approach. A simple methodology based on the definitions of ordered and chaotic dynamical systems was used to determine the separation and fading memory properties of the architecture. The results show the potential use of this architecture as a reservoir for the on-line processing of time-varying inputs.

Investigation of Cortical Signal Propagation and the Resulting Spatiotemporal Patterns in Memristor-Based Neuronal Network

Complexity is the undeniable part of the natural systems providing them with unique and wonderful capabilities. Memristor is known to be a fundamental block to generate complex behaviors. It also is reported to be able to emulate synaptic long-term plasticity as well as short-term plasticity. Synaptic plasticity is one of the important foundations of learning and memory as the high-order functional properties of the brain. In this study, it is shown that memristive neuronal network can represent plasticity phenomena observed in biological cortical synapses. A network of neuronal units as a two-dimensional excitable tissue is designed with 3-neuron Hopfield neuronal model for the local dynamics of each unit. The results show that the lattice supports spatiotemporal pattern formation without supervision. It is found that memristor-type coupling is more noticeable against resistor-type coupling, while determining the excitable tissue switch over different complex behaviors. The stability of the resulting spatiotemporal patterns against noise is studied as well. Finally, the bifurcation analysis is carried out for variation of memristor effect. Our study reveals that the spatiotemporal electrical activity of the tissue concurs with the bifurcation analysis. It is shown that the memristor coupling intensities, by which the system undergoes periodic behavior, prevent the tissue from holding wave propagation. Besides, the chaotic behavior in bifurcation diagram corresponds to turbulent spatiotemporal behavior of the tissue. Moreover, we found that the excitable media are very sensitive to noise impact when the neurons are set close to their bifurcation point, so that the respective spatiotemporal pattern is not stable.

Efficient and self-adaptive in-situ learning in multilayer memristor neural networks

Memristors with tunable resistance states are emerging building blocks of artificial neural networks. However, in situ learning on a large-scale multiple-layer memristor network has yet to be demonstrated because of challenges in device property engineering and circuit integration. Here we monolithically integrate hafnium oxide-based memristors with a foundry-made transistor array into a multiple-layer neural network. We experimentally demonstrate in situ learning capability and achieve competitive classification accuracy on a standard machine learning dataset, which further confirms that the training algorithm allows the network to adapt to hardware imperfections. Our simulation using the experimental parameters suggests that a larger network would further increase the classification accuracy. The memristor neural network is a promising hardware platform for artificial intelligence with high speed-energy efficiency.

K-means Data Clustering with Memristor Networks.

Memristor-based neuromorphic networks have been actively studied as a promising candidate to overcome the von-Neumann bottleneck in future computing applications. Several recent studies have demonstrated memristor network's capability to perform supervised as well as unsupervised learning, where features inherent in the input are identified and analyzed by comparing with features stored in the memristor network. However, even though in some cases the stored feature vectors can be normalized so that the winning neurons can be directly found by the (input) vector-(stored) vector dot-products, in many other cases, normalization of the feature vectors is not trivial or practically feasible, and calculation of the actual Euclidean distance between the input vector and the stored vector is required. Here we report experimental implementation of memristor crossbar hardware systems that can allow direct comparison of the Euclidean distances without normalizing the weights. The experimental system enables unsupervised K-means clustering algorithm through online learning, and produces high classification accuracy (93.3%) for the standard IRIS data set. The approaches and devices can be used in other unsupervised learning systems, and significantly broaden the range of problems a memristor-based network can solve.

Chimera states and synchronization behavior in multilayer memristive neural networks

To study the effect of electromagnetic induction on the spatiotemporal dynamic behavior of neural networks, in this paper, we have mainly studied both the synchronization behavior and the evolution of chimera states (CS) in coupled neural networks. To do this, a multilayer memristive neural network was constructed by selecting the Hindmarsh–Rose neurons as the network nodes, and the effect of electromagnetic induction is introduced by using the cubic flux-controlled memristive model as synapse. For simplicity, the following coupling scheme is adopted: only the coupling connections for the neurons between different layers are considered with memristive synapses, while those neurons in each layer are still bidirectional coupled with the classical electrical synapses. It is found that, by adjusting the coupled strength of electrical synapses and the parameters of memristive synapses, the coexistence behavior of coherent and incoherent states, i.e., the CS, could appear in each layer. It is interesting that the CS are also found in inter-layer memristive synapse network. Furthermore, we have discussed the synchronization behavior in this multilayer memristive neural network, one can find when the whole multilayer network is in a synchronization state, not only the spatiotemporal consistency of the CS in each layer neural networks is observed, but also the memductance of all memristive synapses is completely synchronized. Our results suggest that the electromagnetic induction may play an important role in regulating the dynamic behavior of neural networks, and the introduction of memristive synapse into the biological neural network will provide useful clues for revealing the memory behavior of the neural system in human brain.

Stochastic Learning in Neuromorphic Hardware via Spike Timing Dependent Plasticity With RRAM Synapses

Hardware processors for neuromorphic computing are gaining significant interest as they offer the possibility of real in-memory computing, thus by-passing the limitations of speed and energy consumption of the von Neumann architecture. One of the major limitations of current neuromorphic technology is the lack of bio-realistic and scalable devices to improve the current design of artificial synapses and neurons. To overcome these limitations, the emerging technology of resistive switching memory has attracted wide interest as a nano-scaled synaptic element. This paper describes the implementation of a perceptron-like neuromorphic hardware capable of spike-timing dependent plasticity (STDP), and its operation under stochastic learning conditions. The learning algorithm of a single or multiple patterns, consisting of either static or dynamic visual input data, is described. The impact of noise is studied with respect to learning efficiency (false fire, true fire) and learning time. Finally, the impact of stochastic learning rule, such as the inversion of the time dependence of potentiation and depression in STDP, is considered. Overall, the work provides a proof of concept for unsupervised learning by STDP in memristive networks, providing insight into the dynamics of stochastic learning and supporting the understanding and design of neuromorphic networks with emerging memory devices.

A wide-range operating synaptic device based on organic ferroelectricity with low energy consumption.

In this work, a wide-range operating synaptic device based on organic ferroelectricity has been demonstrated. The device possesses a simple two-terminal structure by using a ferroelectric phase-separated polymer blend as the active layer and gold/indium tin oxide (ITO) as the top/bottom electrodes, and exhibits a distinctive history-dependent resistive switching behavior at room temperature. And the device with low energy consumption (∼50 fJ μm−2 per synaptic event) can provide a reliable synaptic function of potentiation, depression and the complex memory behavior simulation of differential responses to diverse stimulations. In addition, using simulations, the accuracy of 32 × 32 pixel image recognition is improved from 76.21% to 85.06% in the classical model Cifar-10 with 1024 levels of the device, which is an important step towards the higher performance goal in image recognition based on memristive neuromorphic networks.

A Memristor-Based Optimization Framework for Artificial Intelligence Applications

Memristors have recently received significant attention as device-level components for building a novel generation of computing systems. These devices have many promising features, such as non-volatility, low power consumption, high density, and excellent scalability. The ability to control and modify biasing voltages at memristor terminals make them promising candidates to efficiently perform matrix-vector multiplications and solve systems of linear equations. In this article, we discuss how networks of memristors arranged in crossbar arrays can be used for efficiently solving optimization and machine learning problems. We introduce a new memristor-based optimization framework that combines the computational merits of memristor crossbars with the advantages of an operator splitting method, the alternating direction method of multipliers (ADMM). Here, ADMM helps in splitting a complex optimization problem into subproblems that involve the solution of systems of linear equations. The strength of this framework is shown by applying it to linear programming, quadratic programming, and sparse optimization. In addition to ADMM, implementation of a customized power iteration method for eigenvalue/eigenvector computation using memristor crossbars is discussed. The memristor-based power iteration method can further be applied to principal component analysis. The use of memristor crossbars yields a significant speed-up in computation, and thus, we believe, has the potential to advance optimization and machine learning research in artificial intelligence.