Neuromorphic Processor Research Articles

Event-Based Processing of Single Photon Avalanche Diode Sensors

Single Photon Avalanche Diode sensor arrays operating in direct time of flight mode can perform 3D imaging using pulsed lasers. Operating at high frame rates, SPAD imagers typically generate large volumes of noisy and largely redundant spatio-temporal data. This results in communication bottlenecks and unnecessary data processing. In this work, we propose a neuromorphic processing solution to this problem. By processing the spatio-temporal patterns generated by the SPADs in a local, event-based manner, the proposed $128\times 128$ pixel sensor-processor system reduces the size of output data from the sensor by orders of magnitude while increasing the utility of the output data in the context of challenging recognition tasks. To test the proposed system, the first large scale complex SPAD imaging dataset is captured using an existing $32\times 32$ pixel sensor. The generated dataset consists of 24000 recordings and involves high-speed view-invariant recognition of airplanes with background clutter. The frame-based SPAD imaging dataset is converted via several alternative methods into event-based data streams and processed using the proposed $125\times 125$ receptive field neuromorphic processor as well as a range of feature extractor networks and pooling methods. The output of the proposed event generation methods are then processed by an event-based feature extraction and classification system implemented in FPGA hardware. The event-based processing methods are compared to processing the original frame-based dataset via frame-based but otherwise identical architectures. The results show the event-based methods are superior to the frame-based approach both in terms of classification accuracy and output data-rate.

The Development of Silicon for AI: Different Design Approaches

This paper provides a review of design approaches towards artificial intelligence (AI) System-on-Chip. AI algorithms have progressed over the past decades from perceptron-based neural network (NN) and neuro-fuzzy (NF) system to today's deep neural network (DNN) and neuromorphic computing. Recent DNN hardware accelerators focus on energy-efficient integration of digital circuits to realize real-time DNN operation while neuromorphic processors deploy new memory technologies with analog computation for low power consumption. However, different design approaches can be applied to such processor implementation with their pros and cons. This paper reviews from the early processor designs for NN and NF in both mixed-mode and digital implementations to the recent DNN SoC designs that we have proposed for a decade. The former content deals with NN and NF processors used as a functional building block of a machine vision SoC, while the latter concentrates on integration of the whole DNN function. We also provide a discussion on the approaches, and provide perspective on future research directions.

ACortex: An Energy-Efficient Multipurpose Mixed-Signal Inference Accelerator

We introduce “aCortex,” an extremely energy-efficient, fast, compact, and versatile neuromorphic processor architecture suitable for the acceleration of a wide range of neural network inference models. The most important feature of our processor is a configurable mixed-signal computing array of vector-by-matrix multiplier (VMM) blocks utilizing embedded nonvolatile memory arrays for storing weight matrices. Analog peripheral circuitry for data conversion and high-voltage programming are shared among a large array of VMM blocks to facilitate compact and energy-efficient analog-domain VMM operation of different types of neural network layers. Other unique features of aCortex include configurable chain of buffers and data buses, simple and efficient instruction set architecture and its corresponding multiagent controller, programmable quantization range, and a customized refresh-free embedded dynamic random access memory. The energy-optimal aCortex with 4-bit analog computing precision was designed in a 55-nm process with embedded NOR flash memory. Its physical performance was evaluated using experimental data from testing individual circuit elements and physical layout of key components for several common benchmarks, namely, Inception-v1 and ResNet-152, two state-of-the-art deep feedforward networks for image classification, and GNTM, Google’s deep recurrent network for language translation. The system-level simulation results for these benchmarks show the energy efficiency of 97, 106, and 336 TOp/J, respectively, combined with up to 15 TOp/s computing throughput and 0.27-MB/mm2 storage efficiency. Such estimated performance results compare favorably with those of previously reported mixed-signal accelerators based on much less mature aggressively scaled resistive switching memories.

Area-efficient Transposable Crossbar Synapse Memory Using 6T SRAM Bit Cell for Fast Online Learning of Neuromorphic Processors

We present a transposable crossbar synapse memory using 6T SRAM bit cell to speed up online learning of neuromorphic processors at minimal area cost. Based on the proposed integrated transposable row addressing scheme using a row-transition multiplexer, fast write and read operations are made possible for both row-wise and column-wise accesses in an integrated 6T SRAM bit cell array at much smaller area cost compared to the previous works. A 256×256 4-bit transposable synapse memory was implemented in a 28 nm CMOS technology, which had 26% area overhead against the non-transposable 6T synapse memory. The estimated performance gains for unsupervised learning algorithms of spiking neural network and restricted Boltzmann machine using the MNIST data set were 6.3× and 19.3× respectively compared to the non-transposable synapse memory.

Synaptic Delays for Insect-Inspired Temporal Feature Detection in Dynamic Neuromorphic Processors.

Spiking neural networks are well-suited for spatiotemporal feature detection and learning, and naturally involve dynamic delay mechanisms in the synapses, dendrites, and axons. Dedicated delay neurons and axonal delay circuits have been considered when implementing such pattern recognition networks in dynamic neuromorphic processors. Inspired by an auditory feature detection circuit in crickets, featuring a delayed excitation by post-inhibitory rebound, we investigate disynaptic delay elements formed by inhibitory–excitatory pairs of dynamic synapses. We configured such disynaptic delay elements in the DYNAP-SE neuromorphic processor and characterized the distribution of delayed excitations resulting from device mismatch. Interestingly, we found that the disynaptic delay elements can be configured such that the timing and magnitude of the delayed excitation depend mainly on the efficacy of the inhibitory and excitatory synapses, respectively, and that a neuron with multiple delay elements can be tuned to respond selectively to a specific pattern. Furthermore, we present a network with one disynaptic delay element that mimics the auditory feature detection circuit of crickets, and we demonstrate how varying synaptic weights, input noise and processor temperature affect the circuit. Dynamic delay elements of this kind open up for synapse level temporal feature tuning with configurable delays of up to 100 ms.

Architecture and Design of a Spiking Neuron Processor Core Towards the Design of a Large-scale Event-Driven 3D-NoC-based Neuromorphic Processor

Neuromorphic computing tries to model in hardware the biological brain which is adept at operating in a rapid, real-time, parallel, low power, adaptive and fault-tolerant manner within a volume of 2 liters. Leveraging the event driven nature of Spiking Neural Network (SNN), neuromorphic systems have been able to demonstrate low power consumption by power gating sections of the network not driven by an event at any point in time. However, further exploration in this field towards the building of edge application friendly agents and efficient scalable neuromorphic systems with large number of synapses necessitates the building of small-sized low power spiking neuron processor core with efficient neuro-coding scheme and fault tolerance. This paper presents a spiking neuron processor core suitable for an event-driven Three-Dimensional Network on Chip (3D-NoC) SNN based neuromorphic systems. The spiking neuron Processor core houses an array of leaky integrate and fire (LIF) neurons, and utilizes a crossbar memory in modelling the synapses, all within a chip area of 0.12mm2 and was able to achieves an accuracy of 95.15% on MNIST dataset inference.

Optimal Distribution of Spiking Neurons Over Multicore Neuromorphic Processors

In a multicore neuromorphic processor embedding a learning algorithm, a presynaptic neuron is occasionally located in a different core from the cores of its postsynaptic neurons, which needs neuron-to-target core communication for inference through a network router. The more neuron-to-target core connections, the more workload is imposed on the network router, which the more likely causes event routing congestion. Another significant challenge arising from a large number of neuron-to-core connections is data duplication in multiple cores for the learning algorithm to access the full data to evaluate weight update. This data duplication consumes a considerable amount of on-chip memory while the memory capacity per core is strictly limited. The optimal distribution of neurons over cores is categorized as an optimization problem with constraints, which may allow the discrete Lagrangian multiplier method (LMM) to optimize the distribution. Proof-of-concept demonstrations were made on the distribution of neurons over cores in a neuromorphic processor embedding a learning algorithm. The choice of the learning algorithm was twofold: a simple spike timing-dependent plasticity learning rule and event-driven random backpropagation algorithm, which are categorized as a two- and three-factor learning rule, respectively. As a result, the discrete LMM significantly reduced the number of neuron-to-core connections for both algorithms by approximately 55% in comparison with the number for random distribution cases, implying a 55% reduction in the workload on the network router and a 52.8% reduction in data duplication. The code is available on-line (https://github.com/guhyunkim/Optimize-neuron-distribution).

Real-Time Ultra-Low Power ECG Anomaly Detection Using an Event-Driven Neuromorphic Processor.

Accurate detection of pathological conditions in human subjects can be achieved through off-line analysis of recorded biological signals such as electrocardiograms (ECGs). However, human diagnosis is time-consuming and expensive, as it requires the time of medical professionals. This is especially inefficient when indicative patterns in the biological signals are infrequent. Moreover, patients with suspected pathologies are often monitored for extended periods, requiring the storage and examination of large amounts of non-pathological data, and entailing a difficult visual search task for diagnosing professionals. In this work we propose a compact and sub-mW low power neural processing system that can be used to perform on-line and real-time preliminary diagnosis of pathological conditions, to raise warnings for the existence of possible pathological conditions, or to trigger an off-line data recording system for further analysis by a medical professional. We apply the system to real-time classification of ECG data for distinguishing between healthy heartbeats and pathological rhythms. Multi-channel analog ECG traces are encoded as asynchronous streams of binary events and processed using a spiking recurrent neural network operated in a reservoir computing paradigm. An event-driven neuron output layer is then trained to recognize one of several pathologies. Finally, the filtered activity of this output layer is used to generate a binary trigger signal indicating the presence or absence of a pathological pattern. We validate the approach proposed using a Dynamic Neuromorphic Asynchronous Processor (DYNAP) chip, implemented using a standard 180nm CMOS VLSI process, and present experimental results measured from the chip.

Neural State Machines for Robust Learning and Control of Neuromorphic Agents

Mixed-signal analog/digital neuromorphic circuits are characterized by ultra-low power consumption, real-time processing abilities, and low-latency response times. These features make them promising for robotic applications that require fast and power-efficient computing. However, due to the device mismatch and variability present in these circuits, developing architectures that can perform complex computations in a robust and reproducible manner is quite challenging. In this paper, we present a spiking neural network architecture implemented using these neuromorphic circuits, that enables reliable control of an autonomous agent as well as robust learning and recognition of visual patterns in a noisy real-world environment. While learning is implemented with a software algorithm running with a chip-in-the-loop setup, the inference and motor control processes are implemented exclusively by the neuromorphic processor, situated on the neuromorphic agent. In addition to this processor device, the agent comprises a dynamic vision sensor which produces spikes as it interacts with the environment in real-time. We show how the robust learning and reliable control properties of the system arise out of a recently proposed neural computational primitive denoted as Neural State Machine (NSM). We describe the features of the NSMs used in this context and demonstrate the agent's real-time robust perception and action behavior with experimental results.

A Braitenberg Vehicle Based on Memristive Neuromorphic Circuits

The Braitenberg vehicle as a simple conceptual model to characterize the response behaviors of animals or insects under a stimulus is widely used to develop autonomous vehicles able to adapt to the varying environments. Considerable effort has been devoted to building neuromorphic processors with the software in the vehicles; however, there has been no demonstration of Braitenberg vehicle with neuromorphic hardware so far. Herein, a Braitenberg vehicle with simple memristive neuromorphic circuits is built for the first time. This vehicle exhibits adaptive behaviors in the supervised learning process and is eventually trained to conduct the task of tracking path. Moreover, the memristive circuit in the vehicle demonstrates a very short response latency (≈56 ns) to input sensory information. Herein, an alternative promising solution to build self‐adaptive robots and pave the way for the realization of autonomous robots based on memristive neuromorphic circuits is offered.

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

Spiking neural networks (SNNs) are nature's versatile solution to fault-tolerant, energy-efficient signal processing. To translate these benefits into hardware, a growing number of neuromorphic spiking NN processors have attempted to emulate biological NNs. These developments have created an imminent need for methods and tools that enable such systems to solve real-world signal processing problems. Like conventional NNs, SNNs can be trained on real, domain-specific data; however, their training requires the overcoming of a number of challenges linked to their binary and dynamical nature. This article elucidates step-by-step the problems typically encountered when training SNNs and guides the reader through the key concepts of synaptic plasticity and data-driven learning in the spiking setting. Accordingly, it gives an overview of existing approaches and provides an introduction to surrogate gradient (SG) methods, specifically, as a particularly flexible and efficient method to overcome the aforementioned challenges.

Takens-inspired neuromorphic processor: A downsizing tool for random recurrent neural networks via feature extraction

We describe a new technique which minimizes the amount of neurons in the hidden layer of a random recurrent neural network (rRNN) for time series prediction. Merging Takens-based attractor reconstruction methods with machine learning, we identify a mechanism for feature extraction that can be leveraged to lower the network size. We obtain criteria specific to the particular prediction task and derive the scaling law of the prediction error. The consequences of our theory are demonstrated by designing a Takens-inspired hybrid processor, which extends a rRNN with a priori designed delay external memory. Our hybrid architecture is therefore designed including both, real and virtual nodes. Via this symbiosis, we show performance of the hybrid processor by stabilizing an arrhythmic neural model. Thanks to our obtained design rules, we can reduce the stabilizing neural network's size by a factor of 15 with respect to a standard system.

Parameter Optimization and Learning in a Spiking Neural Network for UAV Obstacle Avoidance Targeting Neuromorphic Processors

The Lobula giant movement detector (LGMD) is an identified neuron of the locust that detects looming objects and triggers the insect's escape responses. Understanding the neural principles and network structure that leads to these fast and robust responses can facilitate the design of efficient obstacle avoidance strategies for robotic applications. Here, we present a neuromorphic spiking neural network model of the LGMD driven by the output of a neuromorphic dynamic vision sensor (DVS), which incorporates spiking frequency adaptation and synaptic plasticity mechanisms, and which can be mapped onto existing neuromorphic processor chips. However, as the model has a wide range of parameters and the mixed-signal analog-digital circuits used to implement the model are affected by variability and noise, it is necessary to optimize the parameters to produce robust and reliable responses. Here, we propose to use differential evolution (DE) and Bayesian optimization (BO) techniques to optimize the parameter space and investigate the use of self-adaptive DE (SADE) to ameliorate the difficulties of finding appropriate input parameters for the DE technique. We quantify the performance of the methods proposed with a comprehensive comparison of different optimizers applied to the model and demonstrate the validity of the approach proposed using recordings made from a DVS sensor mounted on an unmanned aerial vehicle (UAV).

Physics at Boeing runs the gamut from lightning strikes to neuromorphic processors, with a dash of metamaterials

Physicists are uniquely qualified to solve problems that require an understanding of the bigger picture.

Silicon Photonic Circuit Design Using Rapid Prototyping Foundry Process Design Kits

A successful design, fabrication and test of silicon photonic circuits requires design tools, process design kits (PDKs), foundries for fabrication, and test facilities. This paper describes the complete design flow of photonic circuits using rapid-prototyping multiproject wafer foundry processes available in the SiEPIC program. The focus of this paper is on rapid prototyping based on electron beam lithography as an alternative and complementary to what is available via deep-UV lithography-based foundries. We describe in detail the PDK and the use of open-source and commercial tools for the design of optical filters, sensors, neuromorphic photonic processors, optical switches, and discuss test and packaging approaches for these designs. We demonstrate that a “germanium less” process can be used to build small systems featuring photoconductive detectors, electronics, and phase shifters.

Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms

Recurrent neural networks are currently subject to intensive research efforts to solve temporal computing problems. Neuromorphic processors (NPs), composed of networked neuron and synapse circuit models, natively compute in time and offer an ultralow power solution particularly suited to emerging temporal edge-computing applications (wearable medical devices, for example). The most significant roadblock to addressing useful problems with neuromorphic hardware is the difficulty in maintaining healthy network dynamics in recurrent neural networks. In animal nervous systems, this is achieved via a multitude of adaptive homeostatic mechanisms which act over multiple time scales to counteract network instability induced via drift, component failure, or learning processes such as spike-timing dependent plasticity. One such mechanism is neuronal intrinsic plasticity (IP) where a neuron adapts its parameters which govern its excitability to fire around a target rate. The approach employed in state of the art NPs, based on a central volatile memory remotely setting model parameters, critically constrains parameter variety and bandwidth rendering realization of these essential mechanisms impossible. This paper demonstrates how reconfigurable nonvolatile resistive memories can be incorporated into neuron and synapse circuits allowing memory to be truly colocalized with the computational units in the computing fabric and facilitating the realization of massively parallel local plasticity mechanisms in neuromorphic hardware. Exploiting nonconventional programming operations of HfO2 based RRAM (stochastic SET and the RESET random variable), we propose a technologically plausible IP algorithm and demonstrate its use in the case of a recurrent neural network topology whereby the system self-organizes to sustain stable and healthy network dynamics around a target firing rate.

Discrimination of EMG Signals Using a Neuromorphic Implementation of a Spiking Neural Network.

An accurate description of muscular activity plays an important role in the clinical diagnosis and rehabilitation research. The electromyography (EMG) is the most used technique to make accurate descriptions of muscular activity. The EMG is associated with the electrical changes generated by the activity of the motor neurons. Typically, to decode the muscular activation during different movements, a large number of individual motor neurons are monitored simultaneously, producing large amounts of data to be transferred and processed by the computing devices. In this paper, we follow an alternative approach that can be deployed locally on the sensor side. We propose a neuromorphic implementation of a spiking neural network (SNN) to extract spatio-temporal information of EMG signals locally and classify hand gestures with very low power consumption. We present experimental results on the input data stream using a mixed-signal analog/digital neuromorphic processor. We performed a thorough investigation on the performance of the SNN implemented on the chip, by: first, calculating PCA on the activity of the silicon neurons at the input and the hidden layers to show how the network helps in separating the samples of different classes; second, performing classification of the data using state-of-the-art SVM and logistic regression methods and a hardware-friendly spike-based read-out. The traditional algorithm achieved a classification rate of [Formula: see text] and [Formula: see text], respectively, and the spiking learning method achieved [Formula: see text]. The power consumption of the SNN is [Formula: see text], showing the potential of this approach for ultra-low power processing.

MorphIC: A 65-nm 738k-Synapse/mm 2 Quad-Core Binary-Weight Digital Neuromorphic Processor With Stochastic Spike-Driven Online Learning.

Recent trends in the field of neural network accelerators investigate weight quantization as a means to increase the resource- and power-efficiency of hardware devices. As full on-chip weight storage is necessary to avoid the high energy cost of off-chip memory accesses, memory reduction requirements for weight storage pushed toward the use of binary weights, which were demonstrated to have a limited accuracy reduction on many applications when quantization-aware training techniques are used. In parallel, spiking neural network (SNN) architectures are explored to further reduce power when processing sparse event-based data streams, while on-chip spike-based online learning appears as a key feature for applications constrained in power and resources during the training phase. However, designing power- and area-efficient SNNs still requires the development of specific techniques in order to leverage on-chip online learning on binary weights without compromising the synapse density. In this paper, we demonstrate MorphIC, a quad-core binary-weight digital neuromorphic processor embedding a stochastic version of the spike-driven synaptic plasticity (S-SDSP) learning rule and a hierarchical routing fabric for large-scale chip interconnection. The MorphIC SNN processor embeds a total of 2k leaky integrate-and-fire (LIF) neurons and more than two million plastic synapses for an active silicon area of 2.86mm 2 in 65-nm CMOS, achieving a high density of 738k synapses/mm 2. MorphIC demonstrates an order-of-magnitude improvement in the area-accuracy tradeoff on the MNIST classification task compared to previously-proposed SNNs, while having no penalty in the energy-accuracy tradeoff.

Training deep neural networks for binary communication with the Whetstone method

This paper presents a new technique for training networks for low-precision communication. Targeting minimal communication between nodes not only enables the use of emerging spiking neuromorphic platforms, but may additionally streamline processing conventionally. Low-power and embedded neuromorphic processors potentially offer dramatic performance-per-Watt improvements over traditional von Neumann processors, however programming these brain-inspired platforms generally requires platform-specific expertise which limits their applicability. To date, the majority of artificial neural networks have not operated using discrete spike-like communication. We present a method for training deep spiking neural networks using an iterative modification of the backpropagation optimization algorithm. This method, which we call Whetstone, effectively and reliably configures a network for a spiking hardware target with little, if any, loss in performance. Whetstone networks use single time step binary communication and do not require a rate code or other spike-based coding scheme, thus producing networks comparable in timing and size to conventional ANNs, albeit with binarized communication. We demonstrate Whetstone on a number of image classification networks, describing how the sharpening process interacts with different training optimizers and changes the distribution of activity within the network. We further note that Whetstone is compatible with several non-classification neural network applications, such as autoencoders and semantic segmentation. Whetstone is widely extendable and currently implemented using custom activation functions within the Keras wrapper to the popular TensorFlow machine learning framework.

Corrigendum: Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain.

[This corrects the article DOI: 10.3389/fnins.2018.00891.].