IBM TrueNorth Research Articles

Listen to the Brain-Auditory Sound Source Localization in Neuromorphic Computing Architectures.

Conventional processing of sensory input often relies on uniform sampling leading to redundant information and unnecessary resource consumption throughout the entire processing pipeline. Neuromorphic computing challenges these conventions by mimicking biology and employing distributed event-based hardware. Based on the task of lateral auditory sound source localization (SSL), we propose a generic approach to map biologically inspired neural networks to neuromorphic hardware. First, we model the neural mechanisms of SSL based on the interaural level difference (ILD). Afterward, we identify generic computational motifs within the model and transform them into spike-based components. A hardware-specific step then implements them on neuromorphic hardware. We exemplify our approach by mapping the neural SSL model onto two platforms, namely the IBM TrueNorth Neurosynaptic System and SpiNNaker. Both implementations have been tested on synthetic and real-world data in terms of neural tunings and readout characteristics. For synthetic stimuli, both implementations provide a perfect readout (100% accuracy). Preliminary real-world experiments yield accuracies of 78% (TrueNorth) and 13% (SpiNNaker), RMSEs of 41∘ and 39∘, and MAEs of 18∘ and 29∘, respectively. Overall, the proposed mapping approach allows for the successful implementation of the same SSL model on two different neuromorphic architectures paving the way toward more hardware-independent neural SSL.

Coreset: Hierarchical neuromorphic computing supporting large-scale neural networks with improved resource efficiency

Crossbar-based neuromorphic chips promise improved energy efficiency for spiking neural networks (SNNs), but suffer from the limited fan-in/fan-out constraints and resource mapping inefficiency. In this paper, we propose a new hardware mechanism to enable configurable combination of cores, called coreset. Using this hierarchical method, our end-to-end CSM (which stands for the ‘CoreSet Method’) framework efficiently solves the fan-in/fan-out issues and significantly improves the resource efficiency. Experiment results show that CSM can efficiently support complex network structures as well as significantly improving accuracies. Up to 4.6% improvement compared with those achieved by other neuromorphic chips (i.e. IBM TrueNorth and Intel Loihi), on the CIFAR-10, CIFAR-100 and SVHN datasets is achieved, matching the accuracies of state-of-the-art SNN models. In addition, compared with IBM TrueNorth, CSM achieves improvements of up to 18.5×,6.04× and 3.33× in memory efficiency, core efficiency and extrapolated throughput, respectively, thus enabling support for large-scale modern networks (such as VGG). In fact, our method can find optimal core sizes for minimal silicon area. As a proof of concept, we have implemented an FPGA emulation of coreset-supported neuromorphic computing. It achieves up to 7,737× speed-up compared to software simulation, thus not only facilitating SNN structure exploration and verification in a timely manner, but also enabling earlier prototyping for better neuromorphic hardware performance investigation.

RANC: Reconfigurable Architecture for Neuromorphic Computing

Neuromorphic architectures have been introduced as platforms for energy efficient spiking neural network execution. The massive parallelism offered by these architectures has also triggered interest from non-machine learning application domains. In order to lift the barriers to entry for hardware designers and application developers we present RANC: a Reconfigurable Architecture for Neuromorphic Computing, an open-source highly flexible ecosystem that enables rapid experimentation with neuromorphic architectures in both software via C++ simulation and hardware via FPGA emulation. We present the utility of the RANC ecosystem by showing its ability to recreate behavior of the IBM's TrueNorth and validate with direct comparison to IBM's Compass simulation environment and published literature. RANC allows optimizing architectures based on application insights as well as prototyping future neuromorphic architectures that can support new classes of applications entirely. We demonstrate the highly parameterized and configurable nature of RANC by studying the impact of architectural changes on improving application mapping efficiency with quantitative analysis based on Alveo U250 FPGA. We present post routing resource usage and throughput analysis across implementations of Synthetic Aperture Radar classification and Vector Matrix Multiplication applications, and demonstrate a neuromorphic architecture that scales to emulating 259K distinct neurons and 73.3M distinct synapses.

HFNet: A CNN Architecture Co-designed for Neuromorphic Hardware With a Crossbar Array of Synapses.

The hardware-software co-optimization of neural network architectures is a field of research that emerged with the advent of commercial neuromorphic chips, such as the IBM TrueNorth and Intel Loihi. Development of simulation and automated mapping software tools in tandem with the design of neuromorphic hardware, whilst taking into consideration the hardware constraints, will play an increasingly significant role in deployment of system-level applications. This paper illustrates the importance and benefits of co-design of convolutional neural networks (CNN) that are to be mapped onto neuromorphic hardware with a crossbar array of synapses. Toward this end, we first study which convolution techniques are more hardware friendly and propose different mapping techniques for different convolutions. We show that, for a seven-layered CNN, our proposed mapping technique can reduce the number of cores used by 4.9–13.8 times for crossbar sizes ranging from 128 × 256 to 1,024 × 1,024, and this can be compared to the toeplitz method of mapping. We next develop an iterative co-design process for the systematic design of more hardware-friendly CNNs whilst considering hardware constraints, such as core sizes. A python wrapper, developed for the mapping process, is also useful for validating hardware design and studies on traffic volume and energy consumption. Finally, a new neural network dubbed HFNet is proposed using the above co-design process; it achieves a classification accuracy of 71.3% on the IMAGENET dataset (comparable to the VGG-16) but uses 11 times less cores for neuromorphic hardware with core size of 1,024 × 1,024. We also modified the HFNet to fit onto different core sizes and report on the corresponding classification accuracies. Various aspects of the paper are patent pending.

Modeling and Simulation of Asynchrony in Neuromorphic Computing

Neuromorphic computing is a non-von Neumann architecture which is also referred to as artificial neural network and that allows electronic system to function in the same manner as that of the human brain. In this paper we have developed neural core architecture analogous to that of the human brain. Each neural core has its own computational element neuron, memory to store information and local clock generator for synchronous functioning of neuron along with asynchronous input-output port and its port controller. The neuron model used here is a tailor-made of IBM TrueNorth’s neuron block. Our design methodology includes both synchronous and asynchronous circuit in order to build an event-driven neural network core. We have first simulated our design using Neuroph studio in order to calculate the weights and bias value and then used these weights for hardware implementation. With that we have successfully demonstrated the working of neural core using XOR application. It was designed in VHDL language and simulated in Xilinx ISE software.

Sparse Coding Using the Locally Competitive Algorithm on the TrueNorth Neurosynaptic System.

The Locally Competitive Algorithm (LCA) is a biologically plausible computational architecture for sparse coding, where a signal is represented as a linear combination of elements from an over-complete dictionary. In this paper we map the LCA algorithm on the brain-inspired, IBM TrueNorth Neurosynaptic System. We discuss data structures and representation as well as the architecture of functional processing units that perform non-linear threshold, vector-matrix multiplication. We also present the design of the micro-architectural units that facilitate the implementation of dynamical based iterative algorithms. Experimental results with the LCA algorithm using the limited precision, fixed-point arithmetic on TrueNorth compare favorably with results using floating-point computations on a general purpose computer. The scaling of the LCA algorithm within the constraints of the TrueNorth is also discussed.

REMODEL: Rethinking Deep CNN Models to Detect and Count on a NeuroSynaptic System.

In this work, we perform analysis of detection and counting of cars using a low-power IBM TrueNorth Neurosynaptic System. For our evaluation we looked at a publicly-available dataset that has overhead imagery of cars with context present in the image. The trained neural network for image analysis was deployed on the NS16e system using IBM's EEDN training framework. Through multiple experiments we identify the architectural bottlenecks present in TrueNorth system that does not let us deploy large neural network structures. Following these experiments we propose changes to CNN model to circumvent these architectural bottlenecks. The results of these evaluations have been compared with caffe-based implementations of standard neural networks that were deployed on a Titan-X GPU. Results showed that TrueNorth can detect cars from the dataset with 97.60% accuracy and can be used to accurately count the number of cars in the image with 69.04% accuracy. The car detection accuracy and car count (–/+ 2 error margin) accuracy are comparable to high-precision neural networks like AlexNet, GoogLeNet, and ResCeption, but show a manifold improvement in power consumption.

Modular Spiking Neural Circuits for Mapping Long Short-Term Memory on a Neurosynaptic Processor

Due to the distributed and asynchronous nature of neural computation through low-energy spikes, brain-inspired hardware systems offer high energy efficiency and massive parallelism. One such platform is the IBM TrueNorth neurosynaptic system. Recently, TrueNorth compatible representation learning algorithms have emerged, achieving close to the state-of-the-art performance in various data sets. However, its application in temporal sequence processing models, such as recurrent neural networks (RNNs), is still only at the proof of concept level. There is an inherent difficulty in capturing temporal dynamics of an RNN using spiking neurons, which is only exasperated by the hardware constraints in connectivity and synaptic weight resolution. This paper presents a design flow that overcomes these difficulties and maps a special case of recurrent networks called long short-term memory (LSTM) onto a spike-based platform. The framework utilizes various approximation techniques, such as activation discretization, weight quantization, and scaling and rounding, spiking neural circuits that implement the complex gating mechanisms, and a store-and-release technique to enable neuron synchronization and faithful storage. While the presented techniques can be applied to map LSTM to any spiking neural network (SNN) simulator/emulator, here we choose the TrueNorth chip as the target platform by adhering to its hardware constraints. Three LSTM applications, parity check, extended Reber grammar, and question classification, are evaluated. The tradeoffs among accuracy, performance, and energy tradeoffs achieved on TrueNorth are demonstrated. This is compared with the performance on an SNN platform without hardware constraints, which represents the upper bound of the achievable accuracy.

Semantic Labeling Using a Low-Power Neuromorphic Platform

Deep learning is a powerful technique for the analysis of remote sensing imagery. For applications that require real-time processing on mobile platforms, a low power consumption processing unit is advantageous. The human brain is remarkably powerful at image recognition tasks while operating at very low power consumption levels. Neuromorphic computing designs aim to achieve energy efficiency through the use of spiking neurons and low-precision synapses to perform data processing. We demonstrate here the classification of red, green, blue and depth and hyperspectral data sets using a neuromorphic processing unit (IBM TrueNorth Neurosynaptic System). The convolutional neural-network architecture of the classifier network has been adapted to fit the neuromorphic architecture. The results on overhead imagery and hyperspectral imagery data show that neuromorphic platforms can achieve the state-of-the-art performance in semantic labeling with significantly ( $\approx 1000\times $ ) lower power consumption than traditional GPU-based solutions.

Spiking Optical Flow for Event-Based Sensors Using IBM's TrueNorth Neurosynaptic System.

This paper describes a fully spike-based neural network for optical flow estimation from dynamic vision sensor data. A low power embedded implementation of the method, which combines the asynchronous time-based image sensor with IBM's TrueNorth Neurosynaptic System, is presented. The sensor generates spikes with submillisecond resolution in response to scene illumination changes. These spike are processed by a spiking neural network running on TrueNorth with a 1-ms resolution to accurately determine the order and time difference of spikes from neighbouring pixels, and therefore infer the velocity. The spiking neural network is a variant of the Barlow Levick method for optical flow estimation. The system is evaluated on two recordings for which ground truth motion is available, and achieves an average endpoint error of 11% at an estimated power budget of under 80mW for the sensor and computation.

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Hardware accelerations of deep learning systems have been extensively investigated in industry and academia. The aim of this paper is to achieve ultra-high energy efficiency and performance for hardware implementations of deep neural networks (DNNs). An algorithm-hardware co-optimization framework is developed, which is applicable to different DNN types, sizes, and application scenarios. The algorithm part adopts the general block-circulant matrices to achieve a fine-grained tradeoff of accuracy and compression ratio. It applies to both fully-connected and convolutional layers and contains a mathematically rigorous proof of the effectiveness of the method. The proposed algorithm reduces computational complexity per layer from O(n2) to O(n log n) and storage complexity from O(n2) to O(n), both for training and inference. The hardware part consists of highly efficient Field Programmable Gate Array (FPGA)-based implementations using effective reconfiguration, batch processing, deep pipelining, resource re-using, and hierarchical control. Experimental results demonstrate that the proposed framework achieves at least 152X speedup and 71X energy efficiency gain compared with IBM TrueNorth processor under the same test accuracy. It achieves at least 31X energy efficiency gain compared with the reference FPGA-based work.

(Vittorio de Nora Award Address) New Tools for Brain Research

At the system level neuroscience is trying to understand how neural circuits work in learning, memory, multisensory integration, motor coordination and how the electrochemical function of these circuits is compromised in the case of disease. Diseases that involve the nervous system are highly complex and mostly not well understood. The Global Burden of Disease Study 2010 (GBD 2010), estimated that a substantial proportion of the world’s disease burden came from mental, neurological and substance use disorders. We need new tools and methods at the system level to effectively tackle these conditions and to better elucidate brain function and disease. One such tool is a novel cognitive computing platform, IBM's TrueNorth chip, that enables the use of deep learning techniques in an ultra-low power environment. We have used deep learning with a convolutional neural network (CNN) and TrueNorth technologies for real-time analysis of brain-activity data at the point of sensing. This approach has the potential to create true closed-loop insights (analytics) and therapies for next generation of wearable and implantable devices at the intersection of neurobiology and artificial intelligence. Neurotransmitters are small proteins secreted between neurons to facilitate neural communication. These compounds are key to information processing during behavior. However, until recently, this chemical communication had not been characterized because biosensors suitable to monitor sub-second chemical events in micron dimensions were unavailable. Cyclic voltammetry at carbon-fiber microelectrodes provides measurements with sub-second time resolution and has been used to examine the dynamics of neurotransmitter concentrations. To enhance the sensitivity and the selectivity of these measurements, we have developed polymeric coatings for bare carbon electrodes. Dopamine, serotonin and adenosine are neurotransmitters that were measured with this methodology. Another tool that we have developed is a novel nanoscale electrode array with superior sensitivity and improved spatial resolution which can be used in the future to gain improved understanding of dopamine dysregulation. We report on the scalable fabrication of dopamine neurochemical probes of a nanostructured glassy carbon that is smaller than any existing dopamine sensor and arrays of more than 6000 nanorod probes. Compared with a flat glassy carbon surface, the nanostructured glassy carbon nanorods provide about 7X higher signal per unit area (current density) for dopamine sensing. The glassy carbon nanorods were fabricated by pyrolysis of a lithographically defined polymeric nanostructure with an industry standard semiconductor fabrication infrastructure. The scalable fabrication strategy offers the potential to integrate these nanoscale dopamine rods with an integrated circuit control system, with other sensors and with different modalities of neural activation.

Computing Generalized Matrix Inverse on Spiking Neural Substrate.

Emerging neural hardware substrates, such as IBM's TrueNorth Neurosynaptic System, can provide an appealing platform for deploying numerical algorithms. For example, a recurrent Hopfield neural network can be used to find the Moore-Penrose generalized inverse of a matrix, thus enabling a broad class of linear optimizations to be solved efficiently, at low energy cost. However, deploying numerical algorithms on hardware platforms that severely limit the range and precision of representation for numeric quantities can be quite challenging. This paper discusses these challenges and proposes a rigorous mathematical framework for reasoning about range and precision on such substrates. The paper derives techniques for normalizing inputs and properly quantizing synaptic weights originating from arbitrary systems of linear equations, so that solvers for those systems can be implemented in a provably correct manner on hardware-constrained neural substrates. The analytical model is empirically validated on the IBM TrueNorth platform, and results show that the guarantees provided by the framework for range and precision hold under experimental conditions. Experiments with optical flow demonstrate the energy benefits of deploying a reduced-precision and energy-efficient generalized matrix inverse engine on the IBM TrueNorth platform, reflecting 10× to 100× improvement over FPGA and ARM core baselines.

A Noise Filtering Algorithm for Event-Based Asynchronous Change Detection Image Sensors on TrueNorth and Its Implementation on TrueNorth.

Asynchronous event-based sensors, or “silicon retinae,” are a new class of vision sensors inspired by biological vision systems. The output of these sensors often contains a significant number of noise events along with the signal. Filtering these noise events is a common preprocessing step before using the data for tasks such as tracking and classification. This paper presents a novel spiking neural network-based approach to filtering noise events from data captured by an Asynchronous Time-based Image Sensor on a neuromorphic processor, the IBM TrueNorth Neurosynaptic System. The significant contribution of this work is that it demonstrates our proposed filtering algorithm outperforms the traditional nearest neighbor noise filter in achieving higher signal to noise ratio (~10 dB higher) and retaining the events related to signal (~3X more). In addition, for our envisioned application of object tracking and classification under some parameter settings, it can also generate some of the missing events in the spatial neighborhood of the signal for all classes of moving objects in the data which are unattainable using the nearest neighbor filter.

TrueNorth-enabled real-time classification of EEG data for brain-computer interfacing.

Brain-computer interfaces are commonly proposed to assist individuals with locked-in syndrome to interact with the world around them. In this paper, we present a pipeline to move from recorded brain signals to real-time classification on a low-power platform, such as IBM's TrueNorth Neurosynaptic System. Our results on a EEG-based hand squeeze task show that using a convolutional neural network and a time preserving signal representation strategy provides a good balance between high accuracy and feasibility in a real-time application. This pathway can be adapted to the management of a variety of conditions, including spinal cord injury, epilepsy and Parkinson's disease.

Spike propagation path planning on IBM TrueNorth neurosynaptic system

The authors report on the implementation of a path planning algorithm on the TrueNorth neurosynaptic system. Their implementation exploits processing in the temporal domain within the architectural constraints of the TrueNorth chip to deduce the optimal path. The optimal path is computed on the TrueNorth chip for grid maps with dimensions as large as 173 × 168 nodes consuming ∼70 mW at an operating voltage of 0.8 V.

Always-On Speech Recognition Using TrueNorth, a Reconfigurable, Neurosynaptic Processor

Deep neural networks (DNN) have been shown to be very effective at solving challenging problems in several areas of computing, including vision, speech, and natural language processing. However, traditional platforms for implementing these DNNs are often very power hungry, which has lead to significant efforts in the development of configurable platforms capable of implementing these DNNs efficiently. One of these platforms, the IBM TrueNorth processor, has demonstrated very low operating power in performing visual computing and neural network classification tasks in real-time. The neuron computation, synaptic memory, and communication fabrics are all configurable, so that a wide range of network types and topologies can be mapped to TrueNorth. This reconfigurability translates into the capability to support a wide range of low-power functions in addition to feed-forward DNN classifiers, including for example, the audio processing functions presented here.In this work, we propose an end-to-end audio processing pipeline that is implemented entirely on a TrueNorth processor and designed to specifically leverage the highly-parallel, low-precision computing primitives TrueNorth offers. As part of this pipeline, we develop an audio feature extractor (LATTE) designed for implementation on TrueNorth, and explore the tradeoffs among several design variants in terms of accuracy, power, and performance. We customize the energy-efficient deep neuromorphic networks structures that our design utilizes as the classifier and show how classifier parameters can trade between power and accuracy. In addition to enabling a wide range of diverse functions, the reconfigurability of TrueNorth enables re-training and re-programming the system to satisfy varying energy, speed, area, and accuracy requirements. The resulting system's end-to-end power consumption can be as low as $14.43\text{mW}$ , which would give up to 100 hours of continuous usage with button cell batteries (CR3023 $1.5\; \text{Whr}$ ) or 450 hours with cellphone batteries (iPhone 6s $6.55\; \text{Whr}$ ).

Deep Artificial Neural Networks and Neuromorphic Chips for Big Data Analysis: Pharmaceutical and Bioinformatics Applications.

Over the past decade, Deep Artificial Neural Networks (DNNs) have become the state-of-the-art algorithms in Machine Learning (ML), speech recognition, computer vision, natural language processing and many other tasks. This was made possible by the advancement in Big Data, Deep Learning (DL) and drastically increased chip processing abilities, especially general-purpose graphical processing units (GPGPUs). All this has created a growing interest in making the most of the potential offered by DNNs in almost every field. An overview of the main architectures of DNNs, and their usefulness in Pharmacology and Bioinformatics are presented in this work. The featured applications are: drug design, virtual screening (VS), Quantitative Structure–Activity Relationship (QSAR) research, protein structure prediction and genomics (and other omics) data mining. The future need of neuromorphic hardware for DNNs is also discussed, and the two most advanced chips are reviewed: IBM TrueNorth and SpiNNaker. In addition, this review points out the importance of considering not only neurons, as DNNs and neuromorphic chips should also include glial cells, given the proven importance of astrocytes, a type of glial cell which contributes to information processing in the brain. The Deep Artificial Neuron–Astrocyte Networks (DANAN) could overcome the difficulties in architecture design, learning process and scalability of the current ML methods.

Mapping Generative Models onto a Network of Digital Spiking Neurons.

Stochastic neural networks such as Restricted Boltzmann Machines (RBMs) have been successfully used in applications ranging from speech recognition to image classification, and are particularly interesting because of their potential for generative tasks. Inference and learning in these algorithms use a Markov Chain Monte Carlo procedure called Gibbs sampling, where a logistic function forms the kernel of this sampler. On the other side of the spectrum, neuromorphic systems have shown great promise for low-power and parallelized cognitive computing, but lack well-suited applications and automation procedures. In this work, we propose a systematic method for bridging the RBM algorithm and digital neuromorphic systems, with a generative pattern completion task as proof of concept. For this, we first propose a method of producing the Gibbs sampler using bio-inspired digital noisy integrate-and-fire neurons. Next, we describe the process of mapping generative RBMs trained offline onto the IBM TrueNorth neurosynaptic processor-a low-power digital neuromorphic VLSI substrate. Mapping these algorithms onto neuromorphic hardware presents unique challenges in network connectivity and weight and bias quantization, which, in turn, require architectural and design strategies for the physical realization. Generative performance is analyzed to validate the neuromorphic requirements and to best select the neuron parameters for the model. Lastly, we describe a design automation procedure which achieves optimal resource usage, accounting for the novel hardware adaptations. This work represents the first implementation of generative RBM inference on a neuromorphic VLSI substrate.