Stochastic Computing Research Articles

A Stochastic Computing Scheme of Embedding Random Bit Generation and Processing in Computational Random Access Memory (SC-CRAM)

Stochastic computing (SC) has emerged as a promising solution for performing complex functions on large amounts of data to meet future computing demands. However, the hardware needed to generate random bit-streams using conventional CMOS based technologies drastically increases the area and delay cost. Area costs can be reduced using spintronics based RNGs, however, this will not alleviate the delay costs since stochastic bit generation is still performed separately from the computation. In this paper, we present an SC method of embedding stochastic bit generation and processing in a computational random access memory array, which we refer to as SC-CRAM. We demonstrate that SC-CRAM is a resilient and low cost method for image processing, Bayesian inference systems, and Bayesian belief networks.

A Markov Chain Framework for Modeling the Statistical Properties of Stochastic Computing Finite-State Machines

A general methodology to derive analytically the statistical properties of Stochastic Computing Finite-State Machines (SFSM) is introduced. The SFSMs, expressed as Moore ones, are modeled using Markov Chains, enabling the derivation in closed form of their output sequences’ statistical properties, including their expected value, their auto-& cross-correlation, their auto-& cross-covariance, their variance and standard deviation as well as their mean squared error. A MC overflow/underflow probability model accompanies the methodology, allowing to calculate analytically the expected number of steps before overflows/underflows, setting the guidelines to select the register’s size that reduces erroneous bits originating from them. In the proposed methodology both the input sequence length and the number of the SFSMs’ states are considered as parameters, accelerating the overall design procedure as the necessity for multiple time-consuming numerical simulations is eliminated. The proposed methodology’s accurate modeling capabilities are demonstrated with its application in two SFSMs selected from the SC literature, while comparisons with the numerical experiments justify its correctness.

Sustainable supply chain design under correlated uncertainty in energy and carbon prices

This paper aims to provide an improvement in the modeling of supply chain designs by incorporating correlated uncertainty among multiple parameters, resulting in a more resilient design. A new methodology to generate forecasts for historically correlated time series, regardless of their underlying probability distributions, is presented and applied to generate scenarios for energy and carbon prices, which historically proved to be correlated. These scenarios are then used in a stochastic computation to obtain a three-echelon supply chain design in Europe maximizing the economic performance. The emissions were monetarized through the incorporation of the European Union cap-and-trade emissions trading system into the model. The social impact of the supply chain network is measured in terms of the direct, indirect and induced jobs it creates, which are proportional to the economic performance. By combining the developed methodology with data mining algorithms, a reduction in the number of required scenarios by more than 90% was achieved. The numerical case study moreover shows that the stochastic design ensures an average reduction of emissions by more than 3 ktons compared to the use of a deterministic approach. In comparison, the computation of a stochastic supply chain design without parameter correlation takes 5 times longer.

Spatial control of skyrmion stabilization energy by low-energy Ga+ ion implantation

Magnetic skyrmions are candidates for information carriers in Brownian and stochastic computers. Developing a technique for fabricating a film with a suitable potential landscape, wherein the information carrier may diffuse freely, is essential for these probabilistic computers. In this study, to build the desired local potential into magnetic films, a 1.2 nm-thick Co-Fe-B film with a 5.2 nm-thick cap layer was irradiated by a focused ion beam (FIB) using Ga+ as the ion source under a low acceleration voltage of 5 keV. The fluences ranged from 0 to 25 × 1012 ions/cm2. Consequently, the critical temperature at which skyrmions appear or disappear is shifted by several 1–10 K depending on the ion fluence. The origin of this effect is discussed by observing the ion implantation profile and the surface sputtering depth using time-of-flight secondary ion mass spectrometry (TOF-SIMS) and atomic force microscopy (AFM). The results of TOF-SIMS measurements show that most of the Ga atoms exist in the Co–Fe–B layer. If all Ga atoms exist in the Co–Fe–B layer, the Ga concentration is 7 × 10−3 at. % after irradiation of 0.8 × 1012 ions/cm2. The AFM results show a sputtered pattern with 0.2 nm depth after irradiation of 16 × 1012 ions/cm2. Finally, the effect of irradiation on the diffusion coefficient was examined. It was determined that small fluences of 1.6 × 1012 and 0.8 × 1012 ions/cm2 can construct a potential barrier controlling skyrmions while maintaining diffusion coefficients as high as 10 μm2/s. The FIB process can be used to draw a circuit of probabilistic computers with skyrmions as information carriers.

Flash-based in-memory computing for stochastic computing in image edge detection

The “memory wall” of traditional von Neumann computing systems severely restricts the efficiency of data-intensive task execution, while in-memory computing (IMC) architecture is a promising approach to breaking the bottleneck. Although variations and instability in ultra-scaled memory cells seriously degrade the calculation accuracy in IMC architectures, stochastic computing (SC) can compensate for these shortcomings due to its low sensitivity to cell disturbances. Furthermore, massive parallel computing can be processed to improve the speed and efficiency of the system. In this paper, by designing logic functions in NOR flash arrays, SC in IMC for the image edge detection is realized, demonstrating ultra-low computational complexity and power consumption (25.5 fJ/pixel at 2-bit sequence length). More impressively, the noise immunity is 6 times higher than that of the traditional binary method, showing good tolerances to cell variation and reliability degradation when implementing massive parallel computation in the array.

Hardware Spiking Neural Networks with Pair-Based STDP Using Stochastic Computing

Hardware Spiking Neural Networks with Pair-Based STDP Using Stochastic Computing

Parallel Stochastic Computing Architecture for Computationally Intensive Applications

Stochastic computing requires random number generators to generate stochastic sequences that represent probability values. In the case of an 8-bit operation, a 256-bit length of a stochastic sequence is required, which results in latency issues. In this paper, a stochastic computing architecture is proposed to address the latency issue by employing parallel linear feedback shift registers (LFSRs). The proposed architecture reduces the latency in the stochastic sequence generation process without losing accuracy. In addition, the proposed architecture achieves area efficiency by reducing 69% of flip-flops and 70.4% of LUTs compared to architecture employing shared LFSRs, and 74% of flip-flops and 58% of LUTs compared to the architecture applying multiple LFSRs with the same computational time.

Molecular Noise in Synaptic Communication.

In synaptic molecular communication (MC), the activation of postsynaptic receptors by neurotransmitter (NTs) is governed by a stochastic reaction-diffusion process. This randomness of synaptic MC contributes to the randomness of the electrochemical downstream signal in the postsynaptic cell, called postsynaptic membrane potential (PSP). Since the randomness of the PSP is relevant for neural computation and learning, characterizing the statistics of the PSP is critical. However, the statistical characterization of the synaptic reaction-diffusion process is difficult because the reversible bi-molecular reaction of NTs with receptors renders the system nonlinear. Consequently, there is currently no model available which characterizes the impact of the statistics of postsynaptic receptor activation on the PSP. In this work, we propose a novel statistical model for the synaptic reaction-diffusion process in terms of the chemical master equation (CME). We further propose a novel numerical method which allows to compute the CME efficiently and we use this method to characterize the statistics of the PSP. Finally, we present results from stochastic particle-based computer simulations which validate the proposed models. We show that the biophysical parameters governing synaptic transmission shape the autocovariance of the receptor activation and, ultimately, the statistics of the PSP. Our results suggest that the processing of the synaptic signal by the postsynaptic cell effectively mitigates synaptic noise while the statistical characteristics of the synaptic signal are preserved. The results presented in this paper contribute to a better understanding of the impact of the randomness of synaptic signal transmission on neuronal information processing.

Multilevel emulation for stochastic computer models with application to large offshore wind farms

Abstract Renewable energy projects, such as large offshore wind farms, are critical to achieving low-emission targets set by governments. Stochastic computer models allow us to explore future scenarios to aid decision making while considering the most relevant uncertainties. Complex stochastic computer models can be prohibitively slow, and thus an emulator may be constructed and deployed to allow for efficient computation. We present a novel heteroscedastic Gaussian Process emulator that exploits cheap approximations to a stochastic offshore wind farm simulator. We also conduct a probabilistic sensitivity analysis to understand the influence of key parameters in the wind farm model, which will help us to plan a probability elicitation in the future.

Stochastic schemes for the perturbation of the atmospheric boundary conditions in ocean general circulation models

Advancing the representation of uncertainties in ocean general circulation numerical models is required for several applications, ranging from data assimilation to climate monitoring and extended-range prediction systems. The atmospheric forcing represents one of the main uncertainty sources in numerical ocean models. Here, we formulate and revise different approaches to perturb the air-sea fluxes used within the atmospheric boundary conditions. In particular, perturbation of the fluxes is performed either through i) stochastic modulation of the air-sea transfer coefficients; ii) stochastic modulation of the air-sea flux tendencies; iii) coarse-graining of stochastic sub-grid computation of the fluxes; or iv) multiple bulk formulas. The schemes are implemented and tested in the NEMO4 ocean model, implemented at an eddy-permitting resolution on a domain covering the North Atlantic and Arctic oceans and the Mediterranean Sea. A series of 22-year 4-member ensemble experiments with different stochastic schemes are performed and analyzed for the period 2000-2021, and results are compared in terms of the ensemble mean and, when applicable, ensemble spread of the principal oceanic diagnostics. Results indicate that the schemes, in general, can significantly improve some verification skill scores (e.g. against drifter current speed, SST analyses, and hydrographic profiles) and, in some cases, enhance the mesoscale activity and weaken the large-scale circulation. The response, however, is different depending on the specific scheme, whose choice thus depends on the target application, as detailed in the paper. These findings foster the adoption of these schemes in both extended-range operational ocean forecasts and coupled long-range climate prediction systems, where the boundary conditions perturbations may contribute to performance increases.

FPGA-based implementation of deep neural network using stochastic computing

A serious challenge in artificial real-time applications is the hardware implementation of deep neural networks (DNN). Among various methods, stochastic computing (SC)-based implementations received tremendous attention due to the low hardware overhead. However, the slow convergence rate is a major problem in SC-based neural networks’ implementation, and millions of clock cycles are required to generate a relatively accurate output. The reconfigurability and parallel nature of field programmable gate array (FPGA) chips make them a preferable platform for SC-based DNN implementation. A fully or semi-parallel implementation of DNNs requires extensive hardware resources. In this paper, an efficient method for DNN implementation on an FPGA chip is presented to address these problems. The FPGA chip reconfiguration feature allows a DNN with several different neurons and topologies to be implemented on a single chip. Convergence time is significantly reduced by limiting the length of the stochastic bitstreams and establishing synchronization between the processing units based on precise timing. Furthermore, due to the limited number of input–output pins in the FPGA chip, a sequential architecture is proposed wherein the DNN inputs enter through only three 8-bit ports. This makes it possible to implement DNNs for image-processing applications. The proposed method is implemented using the Verilog hardware description language on the Xilinx FPGA Virtex-7 xc7v2000t chip. The results show a more than 82% reduction in hardware resources and the minimum rate of power consumption compared to state-of-the-art methods. In addition, the average error rate of the implemented DNN is reduced by 2%.

Unconventional computing based on magnetic tunnel junction

The conventional computing method based on the von Neumann architecture is limited by a series of problems such as high energy consumption, finite data exchange bandwidth between processors and storage media, etc., and it is difficult to achieve higher computing efficiency. A more efficient unconventional computing architecture is urgently needed to overcome these problems. Neuromorphic computing and stochastic computing have been considered to be two competitive candidates for unconventional computing, due to their extraordinary potential for energy-efficient and high-performance computing. Although conventional electronic devices can mimic the topology of the human brain, these require high power consumption and large area. Spintronic devices represented by magnetic tunnel junctions (MTJs) exhibit remarkable high-energy efficiency, non-volatility, and similarity to biological nervous systems, making them one of the promising candidates for unconventional computing. In this work, we review the fundamentals of MTJs as well as the development of MTJ-based neurons, synapses, and probabilistic-bit. In the section on neuromorphic computing, we review a variety of neural networks composed of MTJ-based neurons and synapses, including multilayer perceptrons, convolutional neural networks, recurrent neural networks, and spiking neural networks, which are the closest to the biological neural system. In the section on stochastic computing, we review the applications of MTJ-based p-bits, including Boltzmann machines, Ising machines, and Bayesian networks. Furthermore, the challenges to developing these novel technologies are briefly discussed at the end of each section.

Delta Sigma Modulator-Based Dividers for Accurate and Low Latency Stochastic Computing Systems

Stochastic computing (SC) has received considerable research interest in the past decade. Significant efforts have been devoted to reducing computation latency for the stochastic divider, which is the most complex unit in SC. However, current SC systems still lack dividers that can timely operate with other SC units by aligned processing periods. Moreover, all existing stochastic dividers cannot perform accurate division for input values near the center of the SC computation range. This paper proposes two Delta Sigma Modulator (DSM) based stochastic dividers. The proposed first-order DSM-based divider significantly reduces the additional clock cycles needed for division, and also slightly increases the accuracy (e.g., compared with the fastest existing divider of 10-bit resolution, a reduction of 87.5% in the number of additional clock cycles is accomplished, with an average mean square error (MSE) that is decreased from 10−3.9 to 10−4.0). Moreover, a fully compatible second-order DSM-based divider is proposed. It achieves a higher division accuracy (e.g., MSE of 10−4.7 for 10-bit resolution) and does not require additional clock cycles, at the cost of a slightly increased hardware overhead. As an emerging application, SC-based neural networks are implemented as a case study to evaluate the advantages of the proposed designs. The synthesis results show that compared to the network implementation with the most efficient existing stochastic divider, the use of the proposed dividers reduces the total hardware overhead of the network by 32.0% to 46.6%, and slightly improves the classification accuracy. Overall, the proposed divider designs enable an SC system to operate with aligned timing, so resulting in a better implementation.

Stochastic Computing Design and Implementation of a Sound Source Localization System

Stochastic computing (SC) is an alternative computing paradigm that processes data in the form of uniform bit-streams. SC is fault-tolerant and can compute on small, efficient circuits. However, SC is primarily used in scientific research, and its practical implementations for end-users are rare. Digital sound source localization (SSL) is a useful signal processing technique that locates speakers using multiple microphones. SC has not been integrated into SSL in practice or theory. In this work, for the first time to the best of our knowledge, we implement an SSL algorithm in the stochastic domain and develop a functional SC-based sound source localizer. The practical part of this work shows that the proposed stochastic circuit does not depend on conventional analog-to-digital conversion and can process data in the form of pulse-width-modulated (PWM) signals. The proposed SC design consumes up to 39% less area than the conventional binary design. It can also consume less power depending on the computational accuracy, for example, 6% less power consumption for 3-bit inputs. We propose a new cross-correlation (CC) design based on the state-of-the-art Sobol bit-streams for further area and power saving. The proposed design utilizes a <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MUX</monospace> unit for bit-stream generation. It saves the area footprint up to 64% and the power consumption up to 82% compared to the counter-based SC design of CC, which relies on a comparator for bit-stream generation. The presented stochastic circuits, are not limited to SSL and are readily applicable to other practical applications such as radar ranging, wireless location, sonar direction finding, beamforming, and sensor calibration. The project’s source code is made available for public access.

A Stochastic Computing Sigma-Delta Adder Architecture for Efficient Neural Network Design

A stochastic computing (SC) adder architecture, based on sigma-delta modulation is introduced. The operation principle of the stochastic computing sigma-delta (SCSD) adder is presented and the adder is modeled using Markov Chains resulting in the derivation of output’s first-moment statistics and the demonstration of its fast-convergence. The SCSD’s single-bit output enables the connection to existing Stochastic Finite-State Machines realizing non-linear functions and the formation of cascade processing structures appropriate for efficient realizations of SC artificial neurons. To demonstrate the proposed adder’s efficacy, a SCSD adder-based neuron was designed and used as a building block of a SCSD Multi-Layer Perceptron (MLP). The computational accuracy of the SCSD MLP and the hardware resources it requires are compared to those of the Fixed-Point arithmetic implementation, highlighting the effectiveness and significant area savings of the proposed SC approach. Furthermore, the accuracy and hardware requirements of the proposed SCSD MLP are also compared to those of corresponding SC architectures in the literature.

Digital Implementation of Radial Basis Function Neural Networks Based on Stochastic Computing

Nowadays Internet of Things (IoT) and mobile systems use more and more Machine Learning based solutions, which implies a high computation cost with a low energy consumption. This is causing a revival of interest in unconventional hardware computing methods capable of implementing both linear and nonlinear functions with less hardware overhead than conventional fixed point and floating point alternatives. Particularly, this work proposes a novel Radial Basis Function Neural Network (RBF-NN) hardware implementation based on Stochastic Computing (SC), which applies probabilistic laws over conventional digital gates. Several complex functions design to implement RBF-NN are presented and theoretically analyzed, such as the squared Euclidean distance and the stochastic Gaussian kernel similarity function between input samples and prototypes. The efficiency and performance of the methodology is tested over well-known pattern recognition tasks, including the MNIST dataset. The results show a low-cost methodology in terms of logic resources and power, along with an inherent capability to implement complex functions in a simple way. This methodology enables the implementation of massively parallel large scale RBF-NN with relatively low hardware requirements while maintaining 96.20% accuracy, which is nearly the same for the floating point and fixed point models (96.4% and 96.25%, respectively).

Highly Optimized Hardware Morphological Neural Network Through Stochastic Computing and Tropical Pruning

This work aimed to enhance a previous neural network hardware implementation based on an efficient combination of Stochastic Computing (SC) and Morphological Neural Networks (MNN). This enhancement focused on exploiting the natural ease of morphological neurons to be pruned in order to drastically shrink the hardware resources and increase the compactness of our network. That is why we extended our original hybrid two-layer neural network to classify the MNIST problem, a much more demanding benchmark with about 160,000 trainable parameters. The 92% of the weights of the morphological layer were discarded, allowing a drastic shrinkage of the hardware resources and power dissipation without degrading test accuracy. An extensive comparison with other recent neural networks designs shows that the proposed design achieves significant improvements in terms of energy efficiency, throughput and hardware resources.

Dynamic D2D Multihop Offloading in Multi-Access Edge Computing From the Perspective of Learning Theory in Games

In a D2D-enabled MEC system, devices cooperate in task computation by relaying tasks to servers or providing computation capabilities for users. We investigate how nodes choose the roles to join in the offloading process in a dynamic environment, where mobile devices forming a tree-like multihop network can play relays and intermediate executors earning corresponding economic utility. By mathematically modeling the multihop computation offloading, we formulate the task-flow constrained network-wide utility maximization problem as a potential game. Based on the properties of the potential game, we prove the existence of Nash equilibrium and propose two learning-based algorithms, i.e., myopic best response (MBR-CO) and stochastic learning-based computation offloading (SL-CO), to find the equilibrium point in a distributed manner. Theoretical and simulation results show that MBR-CO is dominant in static scenarios, and SL-CO achieves a high utility and stable performance in dynamic scenarios.

Probabilistic investigation of reinforced concrete structures under fire loading accounting for material and geometric uncertainties

The sensitivity of the structural response to variations of material and cross sectional geometrical parameters is investigated for reinforced concrete structures under fire loading. A relatively low cost 2D corotational layered beam finite element is employed in a reduced latin hypercube sampling framework for this purpose. The structural response is assessed in terms of vertical deflection versus time, failure time and cross sectional stress/strain distribution using standard fire curves. The numerical models represent experimentally tested cases in the literature and operate with experimentally derived parameter sets, when possible. A minimum set of three RVs, the bottom concrete cover thickness, steel and concrete yield stregths, out of the initial full set of 12 are identified that drastically reduces the number of RVs and thus the stochastic computation’s cost, while keeping a reasonable envelop for the results. The relationship between structural behavior, material degradation and data extracted from the cross sectional behavior is also successfully established and used to explain why the random set can be reduced to three parameters.

Design and Analysis of Computer Experiments with both Numeral and Distributional Inputs

Nowadays stochastic computer simulations with both numeral and distributional inputs are widely used to mimic complex systems which contain a great deal of uncertainty. This article studies the design and analysis issues of such computer experiments. First, we provide preliminary results concerning the Wasserstein distance in probability measure spaces. To handle the product space of the Euclidean space and the probability measure space, we prove that, through the mapping from a point in the Euclidean space to the mass probability measure at this point, the Euclidean space can be isomorphic to the subset of the probability measure space, which consists of all the mass measures, with respect to the Wasserstein distance. Therefore, the product space can be viewed as a product probability measure space. We derive formulas of the Wasserstein distance between two components of this product probability measure space. Second, we use the above results to construct Wasserstein distance-based space-filling criteria in the product space of the Euclidean space and the probability measure space. A class of optimal Latin hypercube-type designs in this product space are proposed. Third, we present a Wasserstein distance-based Gaussian process model to analyze data from computer experiments with both numeral and distributional inputs. Numerical examples and real applications to a metro simulation are presented to show the effectiveness of our methods.