Resistive Cross-point Array Research Articles

Design of Resistive Synaptic Array for Implementing On-Chip Sparse Learning

The resistive cross-point array architecture has been proposed for on-chip implementation of weighted sum and weight update operations in neuro-inspired learning algorithms. However, several limiting factors potentially hamper the learning accuracy, including the nonlinearity and device variations in weight update, and the read noise, limited ON/OFF weight ratio and array parasitics in weighted sum. With unsupervised sparse coding as a case study algorithm, this paper employs device-algorithm co-design methodologies to quantify and mitigate the impact of these non-ideal properties on the accuracy. Our analysis shows that the realistic properties in weight update are tolerable, while those in weighted sum are detrimental to the accuracy. With calibration of realistic synaptic behaviors from experimental data, our study shows that the recognition accuracy of MNIST handwriting digits degrades from ∼96 to ∼30 percent. The strategies to mitigate this accuracy loss include 1) redundant cells to alleviate the impact of device variations; 2) a dummy column to eliminate the off-state current; and 3) selector and larger wire width to reduce IR drop along interconnects. The selector also reduces the leakage power in weight update. With improved properties by these strategies, the accuracy increases back to ∼95 percent, enabling reliable integration of realistic synaptic devices in neuromorphic systems.

Evaluating Resistive Memory Devices for Neuromorphic Computing Using Ultrashort Voltage Pulses

A special purpose neural algorithm accelerator based on resistive memory could potentially reduce the energy requirements of neural algorithm implementation by six orders of magnitude over conventional general purpose GPU/CPU hardware (1). The device requirements for such a neural algorithm accelerator depend on attributes including write variability, write linearity, and read noise, as described in reference (2). Devices with multiple resistance states accessible via symmetric voltage pulses of constant magnitude are highly desired for neuromorphic hardware implementations. Tantalum oxide resistive memory devices have been previously shown to have low energy requirements and better control of conductivity during switching in the ultrashort time domain (3). We will report on how employing voltage pulses with ultrashort pulse widths allow for finer control of tantalum oxide based device conductance change per pulse (ΔG/G) unlocking additional neuromorphic states. A parameter space of pulse voltage and time is used to elucidate switching kinetics and switching energy expenditure during device operation. Lastly, the performance of several different resistive memory device structures will be comparatively evaluated for suitability in implementing a neural algorithm accelerator. This work was supported by Sandia National Laboratories’ Laboratory Directed Research and Development (LDRD) Program under the Hardware Acceleration of Adaptive Neural Algorithms Grand Challenge. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. REFERENCES: 1. Kadetotad, Deepak, et al. "Parallel architecture with resistive crosspoint array for dictionary learning acceleration." IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 5.2 (2015): 194-204. 2. S. Agarwal, et. al. “Resistive Memory Device Requirements for a Neural Algorithm Accelerator,” International Joint Conference on Neural Networks, (2016) 3. Strachan, John Paul, et al. “Measuring the switching dynamics and energy efficiency of tantalum oxide memristors” Nanotechnology22.50 (2011): 505402. Figure 1

Physical Unclonable Function Exploiting Sneak Paths in Resistive Cross-point Array

The physical unclonable function (PUF) is a promising innovative hardware security primitive that leverages the inherent randomness in the physical systems to produce unique responses upon the inquiry of challenges, thus the PUF could serve as a fingerprint for device authentication. In this paper, we propose a novel PUF implementation exploiting the sneak paths in the resistive cross-point (X-point) array, as a hardware security primitive. The entanglement of the sneak paths in the X-point array greatly enhances the entropy of the physical system, thereby increasing the space of challenge-response pairs to make a strong PUF. The X-point PUF characteristics, such as uniqueness and reliability, are experimentally evaluated on the fabricated $12 \times 12$ cross-point arrays based on the Pt/HfO x /TiN structure. The measurement results show that the average inter-Hamming distance of the response bits is around 46.2% across 28 different arrays, showing sufficient uniqueness. The measurement results also demonstrate that 0% intra-Hamming distance (or 100% reliability) of the response bits can be maintained more than 7.2 h at 100 °C (or equivalently ten years at 40 °C). This paper demonstrates the feasibility of using X-point PUF as a lightweight and reliable PUF for device authentication.

Demonstration of Convolution Kernel Operation on Resistive Cross-Point Array

Convolution is the key operation in the convolutional neural network, one of the most popular deep learning algorithms. The implementation of the convolution kernel on the resistive cross-point array is different than the implementation of the matrix-vector multiplication in prior works. In this letter, we propose a dimensional reduction of 2-D kernel matrix into 1-D column vector, i.e., a column of the array, and enable the parallel readout of multiple 2-D kernels simultaneously. As a proof-of-concept demonstration, we use the Prewitt kernels to detect both horizontal and vertical edges of the $20 \times 20$ pixels of black-and-white MNIST handwritten digits. The experiments were performed on the fabricated $12 \times 12$ resistive cross-point array based on the Pt/HfO x /TiN structure. The experimental results of the Prewitt kernel operation perfectly matches the simulation results, indicating the feasibility of the proposed implementation methodology of the convolution kernel on resistive cross-point array.

On-Chip Sparse Learning Acceleration With CMOS and Resistive Synaptic Devices

Many recent advances in sparse coding led its wide adoption in signal processing, pattern classification, and object recognition applications. Even with improved performance in state-of-the-art algorithms and the hardware platform of CPUs/GPUs, solving a sparse coding problem still requires expensive computations, making real-time large-scale learning a very challenging problem. In this paper, we cooptimize algorithm, architecture, circuit, and device for real-time energy-efficient on-chip hardware acceleration of sparse coding. The principle of hardware acceleration is to recognize the properties of learning algorithms, which involve many parallel operations of data fetch and matrix/vector multiplication/addition. Today's von Neumann architecture, however, is not suitable for such parallelization, due to the separation of memory and the computing unit that makes sequential operations inevitable. Such principle drives both the selection of algorithms and the design evolution from CPU to CMOS application-specific integrated circuits (ASIC) to parallel architecture with resistive crosspoint array (PARCA) that we propose. The CMOS ASIC scheme implements sparse coding with SRAM dictionaries and all-digital circuits, and PARCA employs resistive-RAM dictionaries with special read and write circuits. We show that 65 nm implementation of the CMOS ASIC and PARCA scheme accelerates sparse coding computation by $394$ and $2140\times$ , respectively, compared to software running on a eight-core CPU. Simulated power for both hardware schemes lie in the milli-Watt range, making it viable for portable single-chip learning applications.

Parallel Architecture With Resistive Crosspoint Array for Dictionary Learning Acceleration

This paper proposes a parallel architecture with resistive crosspoint array. The design of its two essential operations, read and write, is inspired by the biophysical behavior of a neural system, such as integrate-and-fire and local synapse weight update. The proposed hardware consists of an array with resistive random access memory (RRAM) and CMOS peripheral circuits, which perform matrix-vector multiplication and dictionary update in a fully parallel fashion, at the speed that is independent of the matrix dimension. The read and write circuits are implemented in 65 nm CMOS technology and verified together with an array of RRAM device model built from experimental data. The overall system exploits array-level parallelism and is demonstrated for accelerated dictionary learning tasks. As compared to software implementation running on a 8-core CPU, the proposed hardware achieves more than 3000 $\times$ speedup, enabling high-speed feature extraction on a single chip.

Parallel Programming of Resistive Cross-point Array for Synaptic Plasticity

This paper proposes a parallel programming scheme for the cross-point array with resistive random access memory (RRAM). Synaptic plasticity in unsupervised learning is realized by tuning the conductance of each RRAM cell. Inspired by the spike-timing-dependent-plasticity (STDP), the programming strength is encoded into the spike firing rate (i.e., pulse frequency) and the overlap time (i.e., duty cycle) of the pre-synaptic node and post-synaptic node, and simultaneously applied to all RRAM cells in the cross-point array. Such an approach achieves parallel programming of the entire RRAM array, only requiring local information from pre-synaptic and post-synaptic nodes to each RRAM cell. As demonstrated by digital peripheral circuits implemented in 65nm CMOS, the programming time of a 40kb RRAM array is 84ns, indicating 900X speedup as compared to state-of-the-art software approach of sparse coding in image feature extraction.