Neural Processing Elements Research Articles

High area efficiency binary neural processing elements with time-domain signals using SFQ circuits

With the increasing processor performance requirements of convolutional neural networks, we believe that research on superconducting neural networks based on single flux quantum (SFQ) circuits holds significant potential. However, the typical multiply and accumulate operations used in neural networks often lead to a significant increase in circuit area due to the limitations of the integration density of the current SFQ circuit fabrication process. In this work, we propose a binary neural processing element (PE) that utilizes time-domain signal expression. Time-domain signals express multi-bit binary data in time information, which reduces the number of data lines to decrease the circuit area. In the proposed binary neural PE, multiplication is achieved through exclusive-NOR gates and addition is realized by controlling the delay of time-domain signals. We designed and simulated a 3 × 3 convolution circuit using 10 Nb layer process with a critical current density of 10 kA cm−2. Compared to the convolution circuit implemented using conventional SFQ logic, the proposed circuit reduces the number of Josephson junctions by approximately 56% and decreases the area by about 60%.

Quantized Neural Network Synthesis for Direct Logic Circuit Implementation

Hardware acceleration enables neural network (NN) inferencing on edge devices and for high throughput applications. Most approaches use neural processing elements for computation while storing weights in memory blocks. To avoid costly memory access, recent efforts seek direct logic implementation with weights hardwired into the circuit. However, special training strategies are often needed, and they could not maintain accuracy. In contrast, we take a trained and quantized NN as input and synthesize it by Booth encoding and logic sharing, resulting in a hardware accelerator without degrading accuracy. Experiments demonstrate that our method outperforms existing work in area reduction and/or throughput and power efficiency.

Dadu-SV: Accelerate Stereo Vision Processing on NPU

Binocular vision and neural networks (CNNs) are widely seen in modern intelligent vision processing systems, such as robotics, autonomous vehicles, and AR gadgets. However, both the classic semiglobal matching (SGM) and deep CNNs entail substantial computing resource to reach the performance goal. Traditional embedded CPU/graphic processor unit (GPU) cannot simultaneously meet the processing speed and energy requirement, while the specialized circuits dedicated to SGM and CNN processing, respectively, will take considerable hardware and development costs. However, as the popularity of deep learning, neural processing units (NPUs) become prevalent in many embedded and edge devices, which possess high throughput computing power to deal with the matrix operations involved by neural networks. In this work, we attempt to take advantage of the neural processing architectures integrated in SoC chips to accelerate the SGM process, so that the hardware resources will be better utilized instead of investing more resources to create specialized SGM components. Thereby, this letter first deploys SGM on NPU by converting the incompatible operations into the neural-computing flow, and a configurable neural processing element is proposed to flexibly support various vector operation sequences. Then, a hybrid dataflow scheduler and the corresponding hardware modification are introduced to accelerate the cost processing, improving hardware utilization and on-chip memory footprint and access. Our solution runs at 45 fps for an image size of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$640\times 480$ </tex-math></inline-formula> , with 128 disparity levels. The speed-energy efficiency is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$52\times $ </tex-math></inline-formula> better than the GPU (Jetson TX1) solution with negligible additional hardware overhead and accuracy loss.

Logic Synthesis of Binarized Neural Networks for Efficient Circuit Implementation

Neural networks (NNs) are key to deep learning systems. Their efficient hardware implementation is crucial to applications at the edge. Binarized NNs (BNNs), where the weights and output of a neuron are of binary values {−1,+1} (or encoded in {0, 1}), have been proposed. As no multiplier required, BNNs are particularly attractive and suitable for hardware realization. Most prior NN synthesis methods target on hardware architectures with neural processing elements (NPEs), where the weights of a neuron are loaded and the output of the neuron is computed. The load-and-compute method, though area efficient, requires expensive memory access, which deteriorates energy and performance efficiency. In this work we aim at synthesizing BNN layers into dedicated logic circuits. We formulate the corresponding model pruning problem and matrix covering problem to reduce the area and routing cost of BNNs. For model pruning, we propose and compare three strategies at the BNN training stage. For matrix covering, we propose a scalable logic-sharing algorithm. By combining these two methods, experimental results justify the effectiveness of the method in terms of area and net savings on FPGA implementation. Our method provides an alternative implementation of BNNs, and can be applied in combination with NPE-based implementation for area, speed, and power tradeoffs.

TMA: Tera‐MACs/W neural hardware inference accelerator with a multiplier‐less massive parallel processor

SummaryComputationally intensive inference tasks of deep neural networks have brought about a revolution in accelerator architecture, aiming to reduce power consumption as well as latency. The key figure‐of‐merit in hardware inference accelerators is the number of multiply‐and‐accumulation operations per watt (MACs/W); the state‐of‐ the‐art MACs/W, so far, has been several hundreds Giga‐MACs/W. We propose a Tera‐ MACS/W neural hardware inference accelerator (TMA) with 8‐bit activations and scalable integer weights less than 1‐byte. The architecture's main feature is a configurable neural processing element for matrix‐vector operations. The proposed neural processing element uses a multiplier‐less massive parallel processor that works without multipliers, which makes it attractive for energy efficient high‐performance neural network applications. We benchmark our system's latency, power, and performance using Alexnet trained on ImageNet. Finally, we compared our accelerator's throughput and power consumption to that of the prior works. The proposed accelerator outperforms the state‐of‐the‐art counterparts, in terms of the energy and area efficiency, achieving 2.3 TMACs/W@1.0 V on a 28‐nm Virtex‐7 FPGA chip.

A Novel Systolic Parallel Hardware Architecture for the FPGA Acceleration of Feedforward Neural Networks

New chips for machine learning applications appear, they are tuned for a specific topology, being efficient by using highly parallel designs at the cost of high power or large complex devices. However, the computational demands of deep neural networks require flexible and efficient hardware architectures able to fit different applications, neural network types, number of inputs, outputs, layers, and units in each layer, making the migration from software to hardware easy. This paper describes novel hardware implementing any feedforward neural network (FFNN): multilayer perceptron, autoencoder, and logistic regression. The architecture admits an arbitrary input and output number, units in layers, and a number of layers. The hardware combines matrix algebra concepts with serial-parallel computation. It is based on a systolic ring of neural processing elements (NPE), only requiring as many NPEs as neuron units in the largest layer, no matter the number of layers. The use of resources grows linearly with the number of NPEs. This versatile architecture serves as an accelerator in real-time applications and its size does not affect the system clock frequency. Unlike most approaches, a single activation function block (AFB) for the whole FFNN is required. Performance, resource usage, and accuracy for several network topologies and activation functions are evaluated. The architecture reaches 550 MHz clock speed in a Virtex7 FPGA. The proposed implementation uses 18-bit fixed point achieving similar classification performance to a floating point approach. A reduced weight bit size does not affect the accuracy, allowing more weights in the same memory. Different FFNN for Iris and MNIST datasets were evaluated and, for a real-time application of abnormal cardiac detection, a ${\hspace{-0.112em}\times \hspace{-0.112em}}256$ acceleration was achieved. The proposed architecture can perform up to 1980 Giga operations per second (GOPS), implementing the multilayer FFNN of up to 3600 neurons per layer in a single chip. The architecture can be extended to bigger capacity devices or multi-chip by the simple NPE ring extension.

Key features of neural variability emerge from self-organized sequence learning in a deterministic neural network

Cortical responses to identical stimuli show high trial-to-trial variability. This variability is commonly interpreted as resulting from internal noise. However, much of the variability can be explained by the pre-stimulus spontaneous activity [1]. In fact, the contribution of this spontaneous activity to the evoked response is sufficiently strong to bias perceptual decisions [2]. Importantly, spontaneous activity is structurally similar to evoked activity [3] and this similarity may be the result of learning an internal model of the environment during development [4]. Consistent with this idea, spontaneous activity seems to be a superset of possible evoked responses [5] and trial-to-trial variability drops at stimulus onset [6]. At present, it is unclear how these features of neural variability arise in cortical circuits. Here, we show that all of these phenomena emerge in a completely deterministic self-organizing recurrent network (SORN) model [7]. The network consists of recurrently connected excitatory and inhibitory populations of McCulloch-Pitts units. The dynamics are shaped by spike-timing dependent plasticity (STDP) and homeostatic plasticity mechanisms in response to structured input sequences. After a period of self-organization, during which the network learns an internal model of the input sequences, we observe all phenomena mentioned above: evoked responses and perceptual decisions can be predicted from prior spontaneous activity, spontaneous activity outlines the realm of evoked responses, Fano factors drop at stimulus onset, and spontaneous activity closely matches evoked activity patterns. In addition, the network produces the common signs of Poissonian variability in single units. In sum, our model demonstrates that key features of neural variability emerge in a fully deterministic network from self-organized sequence learning via the interaction of STDP and homeostatic plasticity mechanisms. These results suggest that the high trial-to-trial variability of neural responses need not be taken as evidence for noisy neural processing elements. Figure 1 Two example results after self-organization. a) The neural variability drops at stimulus onset. b) Spontaneous and evoked activity become more similar during learning

Low Power Adder based ANN

This paper presents an overview of datapath realizations of the Hardware neural network models which perform massive parallel operations for best results and real time applications. Digital implemented neural models processing element – adder with low power consumption is proposed for real-time multimedia applications. Proposed adder is illustrated in the 23-1 tree layer artificial neural network (ANN). Designs were modeled with Verilog HDL and implemented in FPGA domain by targeting the Virtex 7 device.

Compact Modeling of Single Electron Memory Based on Perceptron Designs

In this work, we present a Single electron random access memories based on perceptron designs used as the basic artificial bio-inspired neural processing element. The operation principles are described and illustrated for the first time by simulations results. Combining the Monte Carlo method with a direct solution of the stationary master equation, we use SIMON simulator and MATLAB for training process. The main goal of this work is to build a multilayer neural network used in recognition and classification using single electron devices. We further provide a write/Erase/ Read states chronogram to provide the key element of our work which is the charge stored in output neuron’s quantum dots.

Theoretical note: the C/T ratio in artificial neural networks

This paper describes computer simulations of the effect of the C/T ratio on acquisition rate in artificial neural networks. The networks consisted of neural processing elements that functioned according to a neurocomputational model whose learning rule is consistent with information on dopaminergic mechanisms of reinforcement. In Simulation 1, three comparisons were made: constant C and variable T, variable C and constant T, and a constant C/T with variable C and T. In the last two comparisons, C was manipulated by changing the probability of reinforcement within the intertrial interval (ITI), in the absence of the conditioned stimulus (CS). Acquisition rate tended to increase with C/T, and the invariant ratio had no effect. In Simulation 2, C was manipulated by changing the ITI, with continuous reinforcement in the presence of the CS and no reinforcements in its absence. Results were comparable to those obtained in Simulation 1. Simulation 3 further explored the effect of the invariant ratio, but with larger absolute values of C and T, which slowed acquisition significantly. The results parallel some experimental findings and theoretical implications of the Gibbon-Balsam model, showing that they can emerge from the moment-to-moment dynamics of a neural-network model. In contrast to that model, however, Simulation 3 suggests that the effect of invariant C/T ratios may be bounded.

Image processing using cellular neural networks based on multi-valued and universal binary neurons

Multi-valued neurons (MVNs) and universal binary neurons (UBNs) are neural processing elements with complex-valued weights and high functionality. It is possible to implement an arbitrary mapping described by a partially-defined multiple-valued function on a single MVN, and an arbitrary mapping described by a partially-defined or fully-defined Boolean function (which does not have to be a threshold function) on a single UBN. Rapidly-converging learning algorithms exist for both types of neurons. Such features of MVNs and UBNs may be used to solve different kinds of problems. One of the most successful applications of MVNs and UBNs is their use as basic neurons in cellular neural networks (CNNs) to solve image processing and image analysis problems.

Temporal classification of Drosophila segmentation gene expression patterns by the multi-valued neural recognition method

In order to reconstruct the establishment of the body pattern over time in Drosophila embryos, we have developed automated methods for detecting the age of an embryo on the basis of knowledge about its gene expression patterns. In this paper we perform temporal classification of confocal images of expression patterns of genes controlling segmentation by means of a neural network based on multi-valued neurons (MVN). MVN are artificial neural processing elements with complex-valued weights and high functionality, which proved to be efficient for solving the image recognition problems. The results obtained by this method confirm its efficiency for image recognition and indicate that the method can detect characteristic features of expression patterns which mark their development over time.

TInMANN - The Integer Markovian Artificial Neural Network for Performing Competitive and Kohonen Learning

A massively parallel, all-digital, stochastic architecture - TIn-MANN - is described which performs competitive and Kohonen types of learning at rates as high as 145,000 training examples per second regardless of network size. While composed of very simple neural processing elements, simulations of TInMANN demonstrate its effectiveness on a number of competitive and Kohonen learning problems.

Dynamics of antagonistic neural processing elements.

Two coupled nonlinear first-order systems whose dynamic behavior reflects the neural states exhibited by a spatially localized population of excitatory and inhibitory nerve cells are described. The dynamics of each constituent neural subpopulation represents a fundamental neural information processing element (PE) of a complex neural system. Phase plane analysis is used in this paper to show how such antagonistic positive acting (excitatory) and negative acting (inhibitory) PEs can generate diverse steady-state and temporal phenomena when the nonlinear system parameters of the PEs are altered. By modifying a selected set of parameters, it is possible to program the positive and negative PEs to exhibit various dynamic attributes such as multiple stable states, transient response behavior and limit-cycle oscillations. These dynamic attributes may be used to perform a variety of useful computational tasks in signal processing and vision systems such as short-term memory (STM), temporal filtering (TF) and pulse frequency modulation (PFM). Computer simulations are presented throughout this paper in order to illustrate these dynamic attributes.

Temporal signal processing with high-speed hybrid analog-digital neural networks

There are many problems which fall into the class of temporal signal processing. These problems have in common the need to relate the temporal properties of their inputs. Conventional solutions to these problems often have high hardware overhead, complex algorithmic solutions, or loss of information through the transformation of temporal properties of the input. To this end, a biologically motivated artificial and neural processing element has been developed. As in biological neurons, processing is time dependent and is implemented using both analog and digital techniques. These characteristics make the PE directly applicable a large class of temporal signal processing problems typically encountered in engineering and science. Multiple aspects of the PE behavior are adjustable, which produces a very wide range of behaviors from simple systems with only a few moderately connected processing elements. The processing element models are custom designed electric circuits based on basic CMOS components and therefore all developed systems can be directly implemented in any standard integrated CMOS technology. The integrated implementation, custom design, and a wide range of adaptable behaviors join to produce a very fast, low-hardware solution to complex spatiotemporal signal processing problems. Seven novel systems based on the hybrid PE are discussed as they relate to commonly encountered temporal signal processing problems.

The design, fabrication, and test of a new VLSI hybrid analog-digital neural processing element

A hybrid analog-digital neural processing element with the time-dependent behavior of biological neurons has been developed. The hybrid processing element is designed for VLSI implementation and offers the best attributes of both analog and digital computation. Custom VLSI layout reduces the layout area of the processing element, which in turn increases the expected network density. The hybrid processing element operates at the nanosecond time scale, which enables it to produce real-time solutions to complex spatiotemporal problems found in high-speed signal processing applications. VLSI prototype chips have been designed, fabricated, and tested with encouraging results. Systems utilizing the time-dependent behavior of the hybrid processing element have been simulated and are currently in the fabrication process. Future applications are also discussed.

Compact Architectures For Adaptive Neural Nets

A compact optical architecture capable of implementing a number of adaptive neural net models is described. Compact architectures do not involve traditional optical imaging systems and are potentially rugged, easily constructed, and scalable. The critical devices of the generic architecture include 1-D electro-optic modulators and detectors to implement the neural processing elements, along with the Pockels readout optical modulator (PROM) to encode the analog weights. The architectural advancements include an input/output compatible method of handing both positive and negative values for the elements of the neuron activation vectors and the synaptic weight matrices during learning and recall. These hybrid architectures are capable of implementing linear and nonlinear associators with adaptive learning algorithms.