Stochastic Spiking Research Articles

Integer Factorization with Stochastic Spiking in Percolating Networks of Nanoparticles.

As growth in global demand for computing power continues to outpace ongoing improvements in transistor-based hardware, novel computing solutions are required. One promising approach employs stochastic nanoscale devices to accelerate probabilistic computing algorithms. Percolating Networks of Nanoparticles (PNNs) exhibit stochastic spiking, which is of particular interest as it meets criteria for criticality which is associated with a range of computational advantages. Here, we show several ways in which spiking PNNs can be used as the core stochastic components of coupled networks that allow successful factorization of integers up to 945. We demonstrate asynchronous operation and show that a single device is sufficient to solve all factorization tasks and to generate multiple solutions simultaneously.

Enhancing the Robustness of Spiking Neural Networks with Stochastic Gating Mechanisms

Spiking neural networks (SNNs) exploit neural spikes to provide solutions for low-power intelligent applications on neuromorphic hardware. Although SNNs have high computational efficiency due to spiking communication, they still lack resistance to adversarial attacks and noise perturbations. In the brain, neuronal responses generally possess stochasticity induced by ion channels and synapses, while the role of stochasticity in computing tasks is poorly understood. Inspired by this, we elaborate a stochastic gating spiking neural model for layer-by-layer spike communication, introducing stochasticity to SNNs. Through theoretical analysis, our gating model can be viewed as a regularizer that prevents error amplification under attacks. Meanwhile, our work can explain the robustness of Poisson coding. Experimental results prove that our method can be used alone or with existing robust enhancement algorithms to improve SNN robustness and reduce SNN energy consumption. We hope our work will shed new light on the role of stochasticity in the computation of SNNs. Our code is available at https://github.com/DingJianhao/StoG-meets-SNN/.

Topological data analysis of the firings of a network of stochastic spiking neurons.

Topological data analysis is becoming more and more popular in recent years. It has found various applications in many different fields, for its convenience in analyzing and understanding the structure and dynamic of complex systems. We used topological data analysis to analyze the firings of a network of stochastic spiking neurons, which can be in a sub-critical, critical, or super-critical state depending on the value of the control parameter. We calculated several topological features regarding Betti curves and then analyzed the behaviors of these features, using them as inputs for machine learning to discriminate the three states of the network.

Sampling-based Bayesian inference in recurrent circuits of stochastic spiking neurons

Two facts about cortex are widely accepted: neuronal responses show large spiking variability with near Poisson statistics and cortical circuits feature abundant recurrent connections between neurons. How these spiking and circuit properties combine to support sensory representation and information processing is not well understood. We build a theoretical framework showing that these two ubiquitous features of cortex combine to produce optimal sampling-based Bayesian inference. Recurrent connections store an internal model of the external world, and Poissonian variability of spike responses drives flexible sampling from the posterior stimulus distributions obtained by combining feedforward and recurrent neuronal inputs. We illustrate how this framework for sampling-based inference can be used by cortex to represent latent multivariate stimuli organized either hierarchically or in parallel. A neural signature of such network sampling are internally generated differential correlations whose amplitude is determined by the prior stored in the circuit, which provides an experimentally testable prediction for our framework.

Leveraging Probabilistic Switching in Superparamagnets for Temporal Information Encoding in Neuromorphic Systems

Brain-inspired computing -leveraging neuroscientific principles underpinning the unparalleled efficiency of the brain in solving cognitive tasks -is emerging to be a promising pathway to solve several algorithmic and computational challenges faced by deep learning today. Nonetheless, current research in neuromorphic computing is driven by our well-developed notions of running deep learning algorithms on computing platforms that perform deterministic operations. In this article, we argue that taking a different route of performing temporal information encoding in probabilistic neuromorphic systems may help solve some of the current challenges in the field. The article considers superparamagnetic tunnel junctions as a potential pathway to enable a new generation of brain-inspired computing that combines the facets and associated advantages of two complementary insights from computational neuroscience – how information is encoded and how computing occurs in the brain. Hardware-algorithm co-design analysis demonstrates 97.41% accuracy of a state-compressed 3-layer spintronics enabled stochastic spiking network on the MNIST dataset with high spiking sparsity due to temporal information encoding.

Neuromorphic computing with hybrid CNN–Stochastic Reservoir for time series WiFi based human activity recognition

Wi-Fi Channel State Information (CSI) based human activity recognition (HAR) which using channel disturbances caused by signal reflection is a novel way of environment sensing and motion recognition. The collected channels characteristics are heavily influenced by the environment, human activity patterns and subject’s weight and height. These signal variations reflected from body components are mainly affected by static multipath effects comprises random noise and behave differently in individuals, and thus an active field of research. To reach further for achieving automated real-time classification, lower computational cost and easy adaptability to hardware are necessary. In this work, a CSI-based HAR with hybrid framework, Convolutional Neural Network (CNN)-Stochastic Reservoir (SR) (CNN-SR) has been proposed, enabling a subject adaptable and more efficient hardware implementation with minimal computational complexity. A subcarrier correlation matrix is first computed and portrayed in image without preprocessing based on the reflection of the raw CSI signal induced by human activities at regular intervals, allowing visual observation of whole pattern changes. The time-based features are subsequently extracted through CNN and these feature arrays are then feed into SR which based on stochastic spiking neural network (SSNN) in simple cycle reservoir architecture for template matching. SR offers attractive power savings over typical von Neumann systems, by doing stochastic computations. The proposed method has also been demonstrated that is capable for HAR based on partially captured signals. The signal pattern of each segment can be observed in a single sight and then employed for person-to-person template recognition. This enables HAR with minimal computational complexity and solving the inter-person variability concerns. The results demonstrate that the proposed CNN-SR achieves impressive performance in recognizing human activities and surpasses existing models with an average accuracy of 93.49%.

How network structure affects the dynamics of a network of stochastic spiking neurons.

Up to now, it still remains an open question about the relation between the structure of brain networks and their functions. The effects of structure on the dynamics of neural networks are usually investigated via extensive numerical simulations, while analytical analysis is always very difficult and thus rare. In this work, we explored the effects of a random regular graph on the dynamics of a neural network of stochastic spiking neurons, which has a bistable region when fully connected. We showed by numerical simulations that as the number of each neuron's neighbors decreases, the bistable region shrinks and eventually seems to disappear, and a critical-like transition appears. In the meantime, we made analytical analysis that explains numerical results. We hope this would give some insights into how structure affects the dynamics of neural networks from a theoretical perspective, rather than merely by numerical simulations.

Noise and spike-time-dependent plasticity drive self-organized criticality in spiking neural network: Toward neuromorphic computing

Self-organized criticality (SoC) may optimize information transmission, encoding, and storage in the brain. Therefore, the underlying mechanism of the SoC provides significant insight for large-scale neuromorphic computing. We hypothesized that noise and stochastic spiking plays an essential role in SoC development in spiking neural networks (SNNs). We demonstrated that under appropriate noise levels and spike-time-dependent plasticity (STDP) parameters, an SNN evolves a SoC-like state characterized by a power-law distribution of neuronal avalanche size in a self-organized manner. Consistent with the physiological findings, the development of SNN was characterized by a transition from a subcritical state to a supercritical state and then to a critical state. Excitatory STDP with an asymmetric time window dominated the early phase of development; however, it destabilized the network and transitioned to the supercritical state. Synchronized bursts in the supercritical state enable inhibitory STDP with a symmetric time window, induce the development of inhibitory synapses, and stabilize the network toward the critical state. This sequence of transitions was observed when the appropriate noise level and STDP parameters were set to the initial conditions. Our results suggest that noise or stochastic spiking plays an essential role in SoC development and self-optimizes SNN for computation. Such neural mechanisms of noise harnessing would offer insight into the development of energy-efficient neuromorphic computing.

The Therapeutic Frequency Profile of Subthalamic Nucleus Deep Brain Stimulation in Rats Is Shaped by Antidromic Spike Failure.

The therapeutic mechanisms of subthalamic nucleus (STN) deep brain stimulation (DBS) may depend on antidromic activation of cortex via the hyperdirect pathway. However, hyperdirect pathway neurons cannot reliably follow high-stimulation frequencies, and the spike failure rate appears to correlate with symptom relief as a function of stimulation frequency. We hypothesized that antidromic spike failure contributes to the cortical desynchronization caused by DBS. We measured in vivo evoked cortical activity in female Sprague Dawley rats and developed a computational model of cortical activation from STN DBS. We modeled stochastic antidromic spike failure to determine how spike failure affected the desynchronization of pathophysiological oscillatory activity in cortex. We found that high-frequency STN DBS desynchronized pathologic oscillations via the masking of intrinsic spiking through a combination of spike collision, refractoriness, and synaptic depletion. Antidromic spike failure shaped the parabolic relationship between DBS frequency and cortical desynchronization, with maximum desynchronization at ∼130 Hz. These findings reveal that antidromic spike failure plays a critical role in mediating the dependency of symptom relief on stimulation frequency.SIGNIFICANCE STATEMENT Deep brain stimulation (DBS) is a highly effective neuromodulation therapy, yet it remains uncertain why conventionally used stimulation frequencies (e.g., ∼130 Hz) are optimal. In this study, we demonstrate a potential explanation for the stimulation frequency dependency of DBS through a combination of in vivo experimental measurements and computational modeling. We show that high-frequency stimulation can desynchronize pathologic firing patterns in populations of neurons by inducing an informational lesion. However, sporadic spike failure at these high frequencies limits the efficacy of the informational lesion, yielding a parabolic profile with optimal effects at ∼130 Hz. This work provides a potential explanation for the therapeutic mechanism of DBS, and highlights the importance of considering spike failure in mechanistic models of DBS.

Stochastic photonic spiking neuron for Bayesian inference with unsupervised learning.

Stochasticity is an inherent feature of biological neural activities. We propose a noise-injection scheme to implement a GHz-rate stochastic photonic spiking neuron (S-PSN). The firing-probability encoding is experimentally demonstrated and exploited for Bayesian inference with unsupervised learning. In a breast diagnosis task, the stochastic photonic spiking neural network (S-PSNN) can not only achieve a classification accuracy of 96.6%, but can also evaluate the diagnosis uncertainty with prediction entropies. As a result, the misdiagnosis rate is reduced by 80% compared to that of a conventional deterministic photonic spiking neural network (D-PSNN) for the same task. The GHz-rate S-PSN endows the neuromorphic photonics with high-speed Bayesian inference for reliable information processing in error-critical scenarios.

Stochasticity in the synchronization of strongly coupled spiking oscillators

Synchronization of electrical oscillators is a crucial step toward practical implementation of oscillator-based and bio-inspired computing. Here, we report the emergence of an unusual stochastic pattern in coupled spiking Mott nanodevices. Although a moderate capacitive coupling results in a deterministic alternating spiking, increasing the coupling strength leads counterintuitively to stochastic disruptions of the alternating spiking sequence. The disruptions of the deterministic spiking sequence are a direct consequence of the small intrinsic stochasticity in electrical triggering of the insulator–metal transition. Although the stochasticity is subtle in individual nanodevices, it becomes dramatically enhanced just in a single pair of coupled oscillators and, thus, dominates the synchronization. This is different from the stochasticity and multimodal coupling, appearing due to collective effects in large oscillator networks. The stochastic spiking pattern in Mott nanodevices results in a discrete inter-spike interval distribution resembling those in biological neurons. Our results advance the understanding of the emergent synchronization properties in spiking oscillators and provide a platform for hardware-level implementation of probabilistic computing and biologically plausible electronic devices.

A shallow hybrid classical–quantum spiking feedforward neural network for noise-robust image classification

Deep Convolutional Neural Network (CNN)-based image classification systems are often susceptible to noise interruption, i.e., minor image noise may significantly impact the outcome. On the contrary, classical Spiking Neural Network (SNN) is known for handling noisy data due to the stochastic and temporal behaviour of the spiking neuron signals. However, it is not always feasible to train the weights of the classical SNN due to stochastic and non-differentiable spiking signals. Recent applications of quantum machine learning to stochastic modelling have predicted that quantum computing would prove to be a significant advantage, hence catapulting the study of quantum and neuromorphic computing to a new level of development. Motivated by these observations, this paper introduces a shallow hybrid classical–quantum spiking feedforward neural network referred to as Spiking Quantum Neural Network (SQNN) for dealing with a robust image classification task in the presence of noise and adversarial attacks. The proposed SQNN offers the inherent capabilities of processing unforeseen noisy test images due to its spatial and temporal information. An efficient variational quantum circuit training with the help of a standard back-propagation algorithm obviates the classical Spiking Time-Dependent Plasticity (STDP) and Spike-Prop algorithms, which are often found inefficient in training the feedforward SNN. The proposed SQNN is extensively tested and benchmarked on the PennyLane Quantum Simulator. Experimental results show that the proposed SQNN model supersedes the feedforward SNN (SFNN), Random Quantum Neural Networks (RQNN), AlexNet and ResNet-18 on unseen test images with added noise from the FashionMNIST, MNIST, KMNIST, CIFAR10, and ImageNet datasets.

An integrate-and-fire approach to Ca2+ signaling-The noise of puffs.

Stochastic spiking is a prominent feature of Ca2+ signaling. The main noise source at the cellular level are puffs from inositol-trisphosphate receptor (IP3R) channel clusters in the membrane of the endoplasmic reticulum (ER). While the random cluster activity has been known for decades, a stringent method to derive the puff noise term acting on the cytosolic Ca2+ concentration is still lacking. We adopt a popular description of neural spike generation from neuroscience, the stochastic integrate-and-fire (IF) model, to describe Ca2+ spiking.

Self-organized collective oscillations in networks of stochastic spiking neurons.

The theory of self-organized bistability (SOB) is the counterpart of self-organized criticality for systems tuning themselves to the edge of bistability of a discontinuous phase transition, rather than to the critical point of a continuous one. As far as we are concerned, there are currently few neural network models that display SOB or rather its non-conservative version, self-organized collective oscillations (SOCO). We show that by slightly modifying the firing function, a stochastic excitatory/inhibitory network model can display SOCO behaviors, thus providing some insights into how SOCO behaviors can be generated in neural network models.

Ion-tunable antiambipolarity in mixed ion–electron conducting polymers enables biorealistic organic electrochemical neurons

Biointegrated neuromorphic hardware holds promise for new protocols to record/regulate signalling in biological systems. Making such artificial neural circuits successful requires minimal device/circuit complexity and ion-based operating mechanisms akin to those found in biology. Artificial spiking neurons, based on silicon-based complementary metal-oxide semiconductors or negative differential resistance device circuits, can emulate several neural features but are complicated to fabricate, not biocompatible and lack ion-/chemical-based modulation features. Here we report a biorealistic conductance-based organic electrochemical neuron (c-OECN) using a mixed ion–electron conducting ladder-type polymer with stable ion-tunable antiambipolarity. The latter is used to emulate the activation/inactivation of sodium channels and delayed activation of potassium channels of biological neurons. These c-OECNs can spike at bioplausible frequencies nearing 100 Hz, emulate most critical biological neural features, demonstrate stochastic spiking and enable neurotransmitter-/amino acid-/ion-based spiking modulation, which is then used to stimulate biological nerves in vivo. These combined features are impossible to achieve using previous technologies.

An integrate-and-fire approach to Ca2+ signaling. Part I: Renewal model

In computational neuroscience integrate-and-fire models capture the spike generation by a subthreshold dynamics supplemented by a simple fire-and-reset rule; they allow for a numerically efficient and analytically tractable description of stochastic single cell as well as network dynamics. Stochastic spiking is also a prominent feature of Ca2+ signaling which suggests to adopt the integrate-and-fire approach for this fundamental biophysical process. The model introduced here consists of two components describing 1) activity of clusters of inositol-trisphosphate receptor channels and 2) dynamics of the global Ca2+ concentrations in the cytosol. The cluster dynamics is given in terms of a cyclic Markov chain, capturing the puff, i.e., the punctuated release of Ca2+ from intracellular stores. The cytosolic Ca2+ concentration is described by an integrate-and-fire dynamics driven by the puff current. For the cyclic Markov chain we derive expressions for the statistics of the interpuff interval, the single-puff strength and the puff current assuming constant cytosolic Ca2+. The latter condition is often well approximated because cytosolic Ca2+ varies much slower than the cluster activity does. Furthermore, because the detailed two-component model is numerically expensive to simulate and difficult to treat analytically, we develop an analytical framework to approximate the driving puff current of the stochastic cytosolic Ca2+ dynamics by a temporally uncorrelated Gaussian noise. This approximation reduces our two-component system to an integrate-and-fire model with a nonlinear drift function and a multiplicative Gaussian white noise, a model that is known to generate a renewal spike train, i.e., a point process with statistically independent interspike intervals. The model allows for fast numerical simulations, permits to derive analytical expressions for the rate of Ca2+ spiking and the coefficient of variation of the interspike interval, and to approximate the interspike interval density and the spike train power spectrum. Comparison of these statistics to experimental data is discussed.

The centrality of population-level factors to network computation is demonstrated by a versatile approach for training spiking networks

Neural activity is often described in terms of population-level factors extracted from the responses of many neurons. Factors provide a lower-dimensional description with the aim of shedding light on network computations. Yet, mechanistically, computations are performed not by continuously valued factors but by interactions among neurons that spike discretely and variably. Models provide a means of bridging these levels of description. We developed a general method for training model networks of spiking neurons by leveraging factors extracted from either data or firing-rate-based networks. In addition to providing a useful model-building framework, this formalism illustrates how reliable and continuously valued factors can arise from seemingly stochastic spiking. Our framework establishes procedures for embedding this property in network models with different levels of realism. The relationship between spikes and factors in such networks provides a foundation for interpreting (and subtly redefining) commonly used quantities such as firing rates.

Mesoscopic description of hippocampal replay and metastability in spiking neural networks with short-term plasticity.

Bottom-up models of functionally relevant patterns of neural activity provide an explicit link between neuronal dynamics and computation. A prime example of functional activity patterns are propagating bursts of place-cell activities called hippocampal replay, which is critical for memory consolidation. The sudden and repeated occurrences of these burst states during ongoing neural activity suggest metastable neural circuit dynamics. As metastability has been attributed to noise and/or slow fatigue mechanisms, we propose a concise mesoscopic model which accounts for both. Crucially, our model is bottom-up: it is analytically derived from the dynamics of finite-size networks of Linear-Nonlinear Poisson neurons with short-term synaptic depression. As such, noise is explicitly linked to stochastic spiking and network size, and fatigue is explicitly linked to synaptic dynamics. To derive the mesoscopic model, we first consider a homogeneous spiking neural network and follow the temporal coarse-graining approach of Gillespie to obtain a "chemical Langevin equation", which can be naturally interpreted as a stochastic neural mass model. The Langevin equation is computationally inexpensive to simulate and enables a thorough study of metastable dynamics in classical setups (population spikes and Up-Down-states dynamics) by means of phase-plane analysis. An extension of the Langevin equation for small network sizes is also presented. The stochastic neural mass model constitutes the basic component of our mesoscopic model for replay. We show that the mesoscopic model faithfully captures the statistical structure of individual replayed trajectories in microscopic simulations and in previously reported experimental data. Moreover, compared to the deterministic Romani-Tsodyks model of place-cell dynamics, it exhibits a higher level of variability regarding order, direction and timing of replayed trajectories, which seems biologically more plausible and could be functionally desirable. This variability is the product of a new dynamical regime where metastability emerges from a complex interplay between finite-size fluctuations and local fatigue.

Characterization and modeling of spiking and bursting in experimental NbO x neuron

Hardware spiking neural networks hold the promise of realizing artificial intelligence with high energy efficiency. In this context, solid-state and scalable memristors can be used to mimic biological neuron characteristics. However, these devices show limited neuronal behaviors and have to be integrated in more complex circuits to implement the rich dynamics of biological neurons. Here we studied a NbO x memristor neuron that is capable of emulating numerous neuronal dynamics, including tonic spiking, stochastic spiking, leaky-integrate-and-fire features, spike latency, temporal integration. The device also exhibits phasic bursting, a property that has scarcely been observed and studied in solid-state nano-neurons. We show that we can reproduce and understand this particular response through simulations using non-linear dynamics. These results show that a single NbO x device is sufficient to emulate a collection of rich neuronal dynamics that paves a path forward for realizing scalable and energy-efficient neuromorphic computing paradigms.

A CMOS-memristor hybrid system for implementing stochastic binary spike timing-dependent plasticity.

This paper describes a fully experimental hybrid system in which a memristive crossbar spiking neural network (SNN) was assembled using custom high-resistance state memristors with analogue CMOS neurons fabricated in 180 nm CMOS technology. The custom memristors used NMOS selector transistors, made available on a second 180 nm CMOS chip. One drawback is that memristors operate with currents in the micro-amperes range, while analogue CMOS neurons may need to operate with currents in the pico-amperes range. One possible solution was to use a compact circuit to scale the memristor-domain currents down to the analogue CMOS neuron domain currents by at least 5–6 orders of magnitude. Here, we proposed using an on-chip compact current splitter circuit based on MOS ladders to aggressively attenuate the currents by over 5 orders of magnitude. This circuit was added before each neuron. This paper describes the proper experimental operation of an SNN circuit using a 1T1R synaptic crossbar together with four post-synaptic CMOS circuits, each with a 5-decade current attenuator and an integrate-and-fire neuron. It also demonstrates one-shot winner-takes-all training and stochastic binary spike-timing-dependent-plasticity learning using this small system.This article is part of the theme issue ‘Advanced neurotechnologies: translating innovation for health and well-being’.