Stochastic Bit Research Articles

An Accuracy/Energy-Flexible Configurable Gabor-Filter Chip Based on Stochastic Computation With Dynamic Voltage–Frequency–Length Scaling

This paper introduces an accuracy/energy-flexible configurable 2-D Gabor filter based on stochastic computation, where stochastic bit stream representing information is used. The Gabor filters show a powerful feature extraction capability, but the calculation based on binary computation is complicated. As opposed to traditional memory-based methods that use fixed Gabor coefficients calculated by software in advance, the proposed circuit dynamically generates the coefficients with small hardware, leading to the power-gating capability. For energy-efficient circuits, dynamic voltage–frequency–length scaling (DVFLS) is proposed to match the performance demands depending on situations. DVFLS controls the lengths of the stochastic bit streams with voltage and frequency, which can lower the energy dissipation and/or increase the throughput with a little accuracy loss. The proposed 64 parallel stochastic Gabor-filter chip is fabricated using TSMC 65-nm CMOS technology with a size of 1.79 mm $\times1.79$ mm. The measurement result shows $4\times $ higher throughput and $4\times $ lower energy than that using a conventional DVFS technique with a 0.391% accuracy loss. Compared with a conventional configurable Gabor filter, the proposed chip achieves a higher throughput/area with more flexibility of the Gabor coefficients.

Stochastic-Based Synapse and Soft-Limiting Neuron with Spintronic Devices for Low Power and Robust Artificial Neural Networks

We propose an innovative stochastic-based computing architecture to implement low-power and robust artificial neural network (S-ANN) with both magnetic tunneling junction (MTJ) and Domain Wall (DW) devices. Our mixed-model HSPICE simulation results have shown that, for a well-known pattern recognition task, a 34-neuron S-ANN implementation achieves more than 1.5 orders of magnitude lower energy consumption and 2.5 orders of magnitude less hidden layer chip area, when compared with its deterministic-based ANN counterparts which are implemented with digital and analog CMOS circuits. We believe that our S-ANN architecture achieves such a remarkable performance gain by leveraging two key ideas. First, because all neural signals are encoded as random bit streams, the standard weighted-sum synapses can be accomplished by stochastic bit writing and reading procedure. Second, we designed and implemented a novel multiple-phase pumping circuit structure to effectively realize the soft-limiting neural transfer function that is essential to improve the overall ANN capability and reduce its network complexity.

Area/Energy-Efficient Gammatone Filters Based on Stochastic Computation

This paper introduces area/energy-efficient gammatone filters based on stochastic computation. The gammatone filter well expresses the performance of human auditory peripheral mechanism and has a potential of improving advanced speech communications systems, especially hearing assisting devices and noise robust speech-recognition systems. Using stochastic computation, a power-and-area hungry multiplier used in a digital filter is replaced by a simple logic gate, leading to area-efficient hardware. However, a straightforward implementation of the stochastic gammatone filter suffers from significantly low accuracy in computation, which results in a low dynamic range (a ratio of the maximum to minimum magnitude) due to a small value of a filter gain. To improve the computation accuracy, gain-balancing techniques are presented that represent the original gain as the product of multiple larger gains introduced at the second-order sections. In addition, dynamic scaling techniques are proposed that scales up small values only on stochastic domain in order to reduce the number of stochastic bits required while maintaining the computation accuracy. For performance comparisons, the proposed stochastic gammatone filters are designed and evaluated on taiwan semiconductor manufacturing company (TSMC) 65-nm CMOS technology. As a result, the proposed filter achieves an area reduction of 90.7% and an energy reduction of 91.8% in comparison with a fixed-point gammatone filter at the same sampling frequency and a comparable dynamic range.

Analog to Stochastic Bit Stream Converter Utilizing Voltage-Assisted Spin Hall Effect

We introducea spintronic analog to stochastic bit stream (SBS) converter (ASC) based on a three terminal magnetic tunnel junction (MTJ) with a heavymetal layer. The critical current of the spin Hall effect (SHE) for switching the MTJ is efficiently modulated by applying a voltage across the MTJ via the voltage-controlled magnetic anisotropy effect. This effect results in the switching probability linearly depending on the amplitude of the analog input signal of the ASC. There are several advantages of using the voltage-assistedSHE for generating an SBS. The stochastic switching behavior of MTJ devices drastically reduces the area overhead by simplifying control circuits in the ASC. Moreover, the voltage-assisted SHE switching improves energy efficiency over traditional spin transfer torque (STT)-based MTJ switching by diminishing ohmic dissipation. Also, multiple MTJs on a single heavy metal layer increases the bandwidth by simultaneously converting analog input signals to SBSs. The performance of the spintronic ASC was evaluated by using the macrospin three terminal MTJ compactmodel integrated into a 45-nm CMOS technology. The proposed ASC can achieve 7x reduction in power consumption comparedto the previousSTT-basedwork and significant improvement in area-efficiency compared with the pure CMOS-based design.

A Reconfigurable Architecture with Sequential Logic-Based Stochastic Computing

Computations based on stochastic bit streams have several advantages compared to deterministic binary radix computations, including low power consumption, low hardware cost, high fault tolerance, and skew tolerance. To take advantage of this computing technique, previous work proposed a combinational logic-based reconfigurable architecture to perform complex arithmetic operations on stochastic streams of bits. The long execution time and the cost of converting between binary and stochastic representations, however, make the stochastic architectures less energy efficient than the deterministic binary implementations. This article introduces a methodology for synthesizing a given target function stochastically using finite-state machines (FSMs), and enhances and extends the reconfigurable architecture using sequential logic. Compared to the previous approach, the proposed reconfigurable architecture can save hardware area and energy consumption by up to 30% and 40%, respectively, while achieving a higher processing speed. Both stochastic reconfigurable architectures are much more tolerant of soft errors (bit flips) than the deterministic binary radix implementations, and their fault tolerance scales gracefully to very large numbers of errors.

On the Robustness of Stochastic Bayesian Machines

This paper revisits the stochastic computing paradigm as a way to implement architectures dedicated to probabilistic inference. In general, it is assumed the operation over stochastic bit streams is robust with respect to radiation transient events effects. Moreover, it can be expected that leveraging the stochastic computing paradigm to implement probabilistic computations such as Bayesian inference implemented in hardware could yield an increased resilience to radiation effects comparatively to deterministic procedures. However, the practical assessment of the robustness against radiation is mandatory before considering stochastic Bayesian machines (SBMs) in hazardous environments. Results of fault injection campaigns at register transfer level provide the first evidences of the intrinsic robustness of SBMs with respect to single event upsets and single event transients.

Logical Computation on Stochastic Bit Streams with Linear Finite-State Machines

Most digital systems operate on a positional representation of data, such as binary radix. An alternative is to operate on random bit streams where the signal value is encoded by the probability of obtaining a one versus a zero. This representation is much less compact than binary radix. However, complex operations can be performed with very simple logic. Furthermore, since the representation is uniform, with all bits weighted equally, it is highly tolerant of soft errors (i.e., bit flips). Both combinational and sequential constructs have been proposed for operating on stochastic bit streams. Prior work has shown that combinational logic can implement multiplication and scaled addition effectively while linear finite-state machines (FSMs) can implement complex functions such as exponentiation and tanh effectively. Prior work on stochastic computation has largely been validated empirically.This paper provides a rigorous mathematical treatment of stochastic implementation of complex functions such as exponentiation and tanh implemented using linear FSMs. It presents two new functions, an absolute value function and exponentiation based on an absolute value, motivated by specific applications. Experimental results show that the linear FSM-based constructs for these functions have smaller area-delay products than the corresponding deterministic constructs. They also are much more tolerant of soft errors.

Generalized Landauer bound as a universal thermodynamic entropy in continuous phase transitions.

The classic Landauer bound can be lowered when erasure errors are permitted. Here we point out that continuous phase transitions characterized by an order parameter can also be viewed as information erasure by resetting a certain number of bits to a standard value. The information-theoretic expression for the generalized Landauer bound in terms of error probability implies thus a universal form for the thermodynamic entropy in the partially ordered phase. We explicitly show that the thermodynamic entropy as a function of interaction parameters and temperature is identical to the information-theoretic expression in terms of error probability alone in the specific example of the Hopfield neural network model of associative memory, a distributed information-processing system of many interacting stochastic bits. In this framework the Landauer bound sets a lower limit for the work associated with "remembering" rather than "forgetting."

A Native Stochastic Computing Architecture Enabled by Memristors

A two-terminal memristor device is a promising digital memory for its high integration density, substantially lower energy consumption compared to CMOS, and scalability below 10 nm. However, a nanoscale memristor is an inherently stochastic device, and extra energy and latency are required to make a deterministic memory based on memristors. Instead of enforcing deterministic storage, we take advantage of the nondeterministic memory for native stochastic computing, where the randomness required by stochastic computing is intrinsic to the devices without resorting to expensive stochastic number generation. This native stochastic computing system can be implemented as a hybrid integration of memristor memory and simple CMOS stochastic computing circuits. We use an approach called group write to program memristor memory cells in arrays to generate random bit streams for stochastic computing. Three methods are proposed to program memristors using stochastic bit streams and compensate for the nonlinear memristor write function: voltage predistortion, parallel single-pulse write, and downscaled write and upscaled read. To evaluate these technical approaches, we show by simulation a memristor-based stochastic processor for gradient descent optimization, and k-means clustering. The native stochastic computing based on memristors demonstrates key advantages in energy and speed in compute-intensive, data-intensive, and probabilistic applications.

Computation on Stochastic Bit Streams Digital Image Processing Case Studies

Maintaining the reliability of integrated circuits as transistor sizes continue to shrink to nanoscale dimensions is a significant looming challenge for the industry. Computation on stochastic bit streams, which could replace conventional deterministic computation based on a binary radix, allows similar computation to be performed more reliably and often with less hardware area. Prior work discussed a variety of specific stochastic computational elements (SCEs) for applications such as artificial neural networks and control systems. Recently, very promising new SCEs have been developed based on finite-state machines (FSMs). In this paper, we introduce new SCEs based on FSMs for the task of digital image processing. We present five digital image processing algorithms as case studies of practical applications of the technique. We compare the error tolerance, hardware area, and latency of stochastic implementations to those of conventional deterministic implementations using binary radix encoding. We also provide a rigorous analysis of a particular function, namely the stochastic linear gain function, which had only been validated experimentally in prior work.

Stochastic BER estimation for coherent QPSK transmission systems with digital carrier phase recovery

We propose a stochastic bit error ratio estimation approach based on a statistical analysis of the retrieved signal phase for coherent optical QPSK systems with digital carrier phase recovery. A family of the generalized exponential function is applied to fit the probability density function of the signal samples. The method provides reasonable performance estimation in presence of both linear and nonlinear transmission impairments while reduces the computational intensity greatly compared to Monte Carlo simulation.

Tracking Forecast Memories for Stochastic Decoding

This paper proposes Tracking Forecast Memories (TFMs) as a novel method for implementing re-randomization and de-correlation of stochastic bit streams in stochastic channel decoders. We show that TFMs are able to achieve decoding performance similar to that of the previous re-randomization methods in the literature (i.e., edge memories), but they exhibit much lower hardware complexity. We then present circuit topologies for analog implementation of TFMs.

A context adaptive bit-plane coder with maximum-likelihood-based stochastic bit-reshuffling technique for scalable video coding

In this paper, we propose a context adaptive bit-plane coding (CABIC) with a stochastic bit reshuffling (SBR) scheme to deliver higher coding efficiency and better subjective quality for fine granular scalable (FGS) video coding. Traditional bit-plane coding in FGS algorithm suffers from poor coding efficiency and subjective quality. To improve coding efficiency, our CABIC constructs context models based on both the energy distribution in a block and the spatial correlations in the adjacent blocks. Moreover, it exploits the context across bit-planes to save side information. To improve subjective quality, our SBR reorders the coefficient bits by their estimated rate-distortion performance. Particularly, we model transform coefficients with Laplacian distributions and incorporate them into the context probability models for content-aware parameter estimation. Moreover, our SBR is implemented with a dynamic priority management that uses a low-complexity dynamic memory organization. Experimental results show that our CABIC improves the PSNR by 0.5~1.0 dB at medium and high bit rates. While maintaining similar or even higher coding efficiency, our SBR improves the subjective quality

Deterministic bit-stream digital neurons

In this paper, we present the design of a deterministic bit-stream neuron, which makes use of the memory rich architecture of fine-grained field-programmable gate arrays (FPGAs). It is shown that deterministic bit streams provide the same accuracy as much longer stochastic bit streams. As these bit streams are processed serially, this allows neurons to be implemented that are much faster than those that utilize stochastic logic. Furthermore, due to the memory rich architecture of fine-grained FPGAs, these neurons still require only a small amount of logic to implement. The design presented here has been implemented on a Virtex FPGA, which allows a very regular layout facilitating efficient usage of space. This allows for the construction of neural networks large enough to solve complex tasks at a speed comparable to that provided by commercially available neural-network hardware.

Learning in Stochastic Bit Stream Neural Networks

This paper presents learning techniques for a novel feedforward stochastic neural network. The model uses stochastic weights and the “bit stream” data representation. It has a clean analysable functionality and is very attractive with its great potential to be implemented in hardware using standard digital VLSI technology. The design allows simulation at three different levels and learning techniques are described for each level. The lowest level corresponds to on-chip learning. Simulation results on three benchmark MONK's problems and handwritten digit recognition with a clean set of 500 16 × 16 pixel digits demonstrate that the new model is powerful enough for the real world applications. Copyright © 1996 Elsevier Science Ltd

Sigmoid neural transfer function realized by percolation

An experiment using the phenomenon of percolation has been conducted to demonstrate the implementation of neural functionality (summing and sigmoid transfer). A simple analog approximation to digital percolation is implemented. The device consists of a piece of amorphous silicon with stochastic bit-stream optical inputs, in which a current percolating from one end to the other defines the neuron output, also in the form of a stochastic bit stream. Preliminary experimental results are presented.

Real time output derivatives for on chip learning using digital stochastic bit stream neurons

The authors present the hardware design of an extremely compact and novel digital stochastic neuron, that has the ability to generate the derivative of its output with respect to an arbitrary input. These derivatives may be used to form the basis of an on chip gradient descent learning algorithm.

Device for generating binary sequences for stochastic computing

A novel technique for the generation of high speed stochastic bit streams in which the ‘1’ density is proportional to a given value is presented. Bit streams of this type are particularly useful in bit serial stochastic computing systems, such as digital stochastic neural networks. The proposed circuitry is highly suitable for VLSI fabrication.