Integrate-and-fire Neuron Research Articles

High-Speed and Low-Power Superconducting Neuromorphic Circuits Based on Quantum Phase-Slip Junctions

The quantum phase-slip junctions (QPSJs) resulting from quantum fluctuation of superconducting order parameter in 1-D superconducting nanowires are explained as exact duals to Josephson junctions (JJs). An overdamped QPSJ can generate a current pulse to emulate a spiking event in biological neurons. We have demonstrated spiking neuron circuits and synaptic circuits using QPSJs in our previous work through simulations in WRspice. The neuron circuit can fire a current pulse as soon as the capacitor voltage is above a threshold, which is similar to an integrate-and-fire neuron (IFN). The synaptic circuit, either based on magnetic JJs and QPSJs or only using QPSJs, functions as an adjustable weighted connection between neurons. To overcome existing problems, such as fan-out limitations, and further explore the possible applications of QPSJs in superconducting neuromorphic computing, we propose new neuromorphic circuits based on QPSJs and JJs in this paper. The neuron, synapse and fan-out circuits operate through the transfer of quantized charge or fluxons, providing circuit operations that emulate brain functions. Since the fabrication of reproducible QPSJ elements remains at a preliminary stage, the demonstration of our circuits design has been primarily done in WRspice simulation with a QPSJ SPICE model.

An AI-based approach in determining the effect of meteorological factors on incidence of malaria

This study presents the classification of malaria-prone zones based on (a) meteorological factors, (b) demographics and (c) patient information. Observations are performed on extended features in dataset over the spiking and non-spiking classifiers including Quadratic Integrate and Fire neuron (QIFN) model as a benchmark. As per research studies, parasite transmission is highly dependent on the (i) stagnant water, (ii) population of area and the (iii) greenery of the locality. Considering these factors, three more attributes were added to the existing novel dataset and comparison on the results is presented. For four feature dataset, QIFN exhibited an accuracy of 97.08% in K10 protocol, and with extended dataset; QIFN yields an accuracy of 99.58% in K10 protocol. The benchmarking results showed reliability and stability. There is 12.47% improvement against multilayer perceptron (MLP) and 5.39% against integrate-and-fire neuron (IFN) model. The QIFN model performed the best over the conventional classifiers for deciphering the risk of acquiring malaria in different geographical regions worldwide.

Achieving Synchronization in Cardiac Oscillators with Suitable Neural Input

The normal cardiac rhythm is the result of collective, synchronized action of a large number of cardiac oscillators which play a crucial role in the determination of the sinus rhythm. The physiological function of the cardiovascular system is under the control of the autonomic nervous system (ANS). The two limbs of the ANS, sympathetic and parasympathetic, are critical in determining the oscillations within the heart. The pumping effectiveness of the heart is controlled by the sympathetic and parasympathetic nerves, which abundantly supply the heart that act in opposing ways. However, the two divisions act together to regulate the activity of the internal organs as per the needs of the body at any particular time. The cardiac centers in the central nervous system exert an influence on the heart’s activity through sympathetic and parasympathetic nerves. This influence governs the rate of beat, the systolic contractile force and the velocity of atrioventricular conduction. The parasympathetic stimulation causes a decrease in heart rate whereas sympathetic stimulation increases it. The intrinsic cardiac nervous system is seen to play an active role in regulating cardiac function which consists of sympathetic and parasympathetic neurons and interconnecting local circuits. An appropriate mathematical model that describes the electrical activity and ion exchange in the sinoatrial node (SAN) is considered and the dynamical equations describing the behaviour of the chosen model are solved with the corresponding source code developed and implemented using Matlab.An Integrate and Fire Neuron (IFN) model is developed that mimics the role of ANS which acts as an external influence to the preferred SAN cell to in order to coax synchronization between the coupled cell pair. The influence of the suitable neuron model in effecting synchrony between the coupled SAN cell pair is demonstrated with the aid of simulation studies.

Nonlinear system identification of receptive fields from spiking neuron data

Identification of a linear filter in cascade with a spiking neuron has been previously considered [1] under the assumption that the input of the Hodgkin-Huxley spiking neuron can be measured in addition to the input and output of the circuit. An alternative method to identify a linear filter given the input to the circuit and the spike time sequence, has been proposed [2] for Integrate-and-Fire (IAF) neurons. The method was further extended [3] to nonlinear receptive fields that can be described by a Volterra series. Here we propose a new system identification methodology to identify a more general NARMAX representation of the nonlinear receptive field in cascade with an ideal-IAF neuron, based only on measurements of the input stimulus and the spike time sequence of the IAF neuron. By using an orthogonal forward selection algorithm we are able to derive the NARMAX representation of a scaled version of the nonlinear filter directly from the reconstructed input of an ideal IAF neuron. The method is further extended to leaky-IAF neurons which require estimating an additional spiking neuron parameter. We use the NARMAX methodology to identify recursively the nonlinear filter and the spiking neuron parameter in the presence of noise. Statistical and dynamical model validation tests are used to check if the identified nonlinear filter models are an adequate representation of the underlying nonlinear information processing mechanisms. The performance and robustness to noise of the proposed methods is demonstrated through numerical simulation studies. Specifically, we compared the generalized (higher-order) frequency response functions (GFRFs) of the original and the identified nonlinear filter (Figures 1 A-D) and evaluated, for different noise levels, the SNR of the model predicted output on a validation data set (Figure 1 E). Figure 1 A&B - GFRF functions of the original filter, C&D - Difference between the original GFRFs and the scaled versions for the identified filter, E - SNR between the validation signal and the identified filter output.

Assessing the role of synchronization and phase coherence in neural communication comparing cortical recordings and integrate-and-fire network models

Cognitive functions likely require that the routes of neural communication can be flexibly modulated. A proposed mechanism for modulating the effective strength of the connections in the neural dynamics relies on band specific neural synchronization and phase relations [1]. Evidence of the modulation of neuronal interactions through the phase relation of rhythmic activity in the gamma band was provided in [2]. Further evidence based on the analysis of a network of integrate-and fire neurons [3] has shown that the phase relation also modulates information transfer and is not specific of the gamma band. Here we combine the study of experimental recordings of local field potentials (LFP’s) and multiple-unit activity (MUA) together with model data to better understand the origins of the phase-dependent modulation of interactions. Recordings include spontaneous activity and natural stimulus-driven activity in the monkey visual cortex V1 [4], as well as natural-stimulus driven activity in monkey auditory cortex [5]. Simulations were obtained extending the recurrent network of integrate-and-fire neurons used in [6] to model the connectivity between two different brain areas. We address some open questions regarding the generation, generality, and mechanistic nature of the phase-dependent modulation. We obtained, for any frequency band, the instantaneous power in each area (reflecting the local neural synchronization), and the instantaneous phases. We analyzed how the power correlation is modulated by the phase relation with a 1ms resolution, in contrast to the hundreds of milliseconds in [2]. We found that this modulation is accompanied by changes in the magnitude of the power of each area separately. Accordingly, we evaluated the role of the power determining the degree of phase coherence and thus the existence of a preferred phase relation. We found that the optimal phase relation associated with maximal power correlation always corresponds to the preferred phase relation for large powers. These results are not frequency band specific and were reproduced with model data, using both unidirectional and bidirectional connections, as well as both excitatory-excitatory and excitatory-inhibitory connections. Our analysis suggests that the degree of local neural synchronization that determines the power of a given rhythm in each of the interacting areas plays a role to be considered together with the one of the phase relations as part of the mechanisms that modulate dynamically the effective strength of the connections.

Model of a sparse encoding neuron

Neurons as Time Encoding Machines (TEMs) have been proposed to capture the information present in sensory stimuli and to encode it into spike trains [1]. These neurons, however, produce spikes at firing rates above Nyquist, which is usually much higher than the amount of information actually present in stimuli. We propose a low-rate neuron which exploits the sparsity or compressibility present in natural signals to produce spikes at a firing rate proportional to the amount of information present in the signal rather than its duration. We consider the IAF (Integrate-and-Fire) neuron circuit as presented in [1], provide appropriate modifications to convert it into a low-rate encoder and develop an algorithm for reconstructing the input stimulus using Compressive Sampling (CS) techniques. The class of input signals is assumed to be a mixture of periodic waveforms, consistent with the brain mechanism of generating and entraining oscillations at multiple frequencies (S in number) simultaneously. The LowRate IAF neuron circuit uses fixed thresholds (δ) as opposed to random thresholds used in [1]. The randomness in inter-spike-interval exhibited in spike trains is produced by an additional component that switches off the IAF circuit (mimicking the “absolute refractory” period) for a random amount of time (with mean μ) after each spike (see figure ​figure1A).1A). We compare the performance of our LowRate neuron firing at spike-rate K (which is determined by the parameters δ and μ) with IAF neurons in [1] operating at and above Nyquist rate N (>K). Because we inject additive white Gaussian noise into the input signal, we use the traditional measure of signal-to-noise ratio (SNR) as our performance metric. The recovery method developed is a greedy pursuit algorithm similar to the one described in [2]. Figure ​Figure1B1B plots the mean output SNR vs. input SNR for a signal with S = 10 frequencies and sparse-encoding ratio K/N = 0.3052. The LowRate IAF neuron (even when operating at about one third the Nyquist rate in this example) outperforms the IAF neurons operating at and above Nyquist rates. Figure ​Figure1C1C (for signals with S = 60) demonstrates that an increase in sparse-encoding ratio K/N improves the performance of LowRate IAF neuron. We are currently extending this methodology to signals sparse in other domains as well. Figure 1 CS IAF neuron A) circuit; output SNR vs. input SNR for B) signals with S = 10 C) signals with S = 60.

A computational model of posterior parietal circuits during decision making and sequential planning

In the posterior parietal cortex (PPC), three anatomically separated but reciprocally interconnected subareas, the lateral intraparietal area (LIP), parietal reach region (PRR) and dorsal area 5 (area 5d), are involved in sensorimotor transformation during planning of eye/arm movements. We have previously designed the saccade/reach effector choice and sequential reach paradigms to dissociate cognitive and executive planning activity in PPC. During saccade/reach choice, LIP and PRR encodes potential saccade/reach plans in parallel before effector decision is formed, and the selected saccade/reach plans thereafter [1], whereas area 5d only reflects the decision outcome [2]. During sequential reach, PRR encodes both immediate and subsequent goals [3], whereas area 5d only represents the immediate upcoming reach [4]. Based on the experimental results above, we propose a network model to conceive the functional connectivity across these areas. The model is composed of an upstream layer representing the abstract motor intentions and a down-stream layer representing concrete motor commands. The up-stream layer consists of LIP and PRR neurons preferring different directions for saccade and reach, respectively, and two populations compete against each other. The down-stream layer is composed of area 5d neurons preferring arm movements to different directions. In each neuronal pool, each leaky integrate-andfire neuron excites others representing nearby targets but inhibits the remaining. The up-stream layer also feed-forwards to the down-stream layer to excite the neurons preferring the same effector and nearby target but to inhibit others. Furthermore, each LIP and PRR neuron receives self-excitation whose magnitudes are modulated by the rule-based behavioral context, reflecting the topdown feedback. Our simulation results show that, during the effector choice and instructed tasks, the effector information is ambiguous at first so that both LIP and PRR show strong responses and counterbalance the total input to area 5d. Once effector information is determined by either choice or instruction, LIP and PRR activities differentiate to trigger or suppress area 5d activity. During the classical memory saccade/reach tasks, effector information is given at the cue onset so that LIP and PRR activities began to differentiate and area 5d fires immediately. During the sequential reaching task, after two targets are simultaneously presented, the activity of the PRR encoding the 1st goal increases faster while the activity of the PRR encoding the 2nd goal increases slower but lasts longer due to the different recurrent connections modulated by the behavioral context. Consequentially, only the activity of the area 5d neuron encoding the 1st goal is elevated before the GO signal. Once the first reach is completed, the area 5d neuron preferring the 2nd goal begins to fire because top-down context information (efference copy in this case) reduces the strength of recurrent connection for the PRR neuron coding the 1st goal. Preliminary results indicate that our model captures all characteristics of neural activity we have observed so far. Our ongoing modeling work will help reveal the network mechanisms underlying the operation of the parietal sensorimotor circuits.

Optimal neural coding in networks of heterogeneous neurons

Neural systems display a prominent level of heterogeneity in synaptic, neuron, and network properties. Even among same-class neurons, differences in excitability properties, for instance, may be present in the system, and this could have strong implications for information processing. A well-known example is the so called population coding ,a strategy which relies on the particular responses of individual neurons of a network to code information. This occurs for instance in V1 cortical networks, where the orientation preference of neurons allows for efficient information coding [1]. However, not many studies have addressed the role of neural heterogeneity in other neural coding strategies. Interestingly, cell-to-cell differences have been recently found to be relevant in synchronization of inhibitory networks [2], coherent activity in electrically coupled neurons [3], synchronized bursting events [4], global detection of weak signals [5], and efficient coding for communication signals [6], highlighting a possible role of heterogeneity on neural information processing. Still, neither the effect of heterogeneity on the dynamics of neural populations nor its influence on neural coding are yet fully understood. In this work, we present a theoretical and numerical study of how different strategies of neural coding (concretely, rate coding and temporal coding) are affected by the presence of heterogeneity in the properties of the neurons in the network, paying special attention to their ability to process and codify incoming signals. We carry out this study in two different systems: the first one, which is conceptually simpler, is a fully-connected network of excitatory integrate-and-fire neurons with small synaptic delays. The second one is more realistic, and consists in a sparsely connected network of excitatory and inhibitory integrateand-fire neurons with small synaptic delays. For both systems, a certain level of heterogeneity is introduced in the neuron firing thresholds, that is, each neuron in the network has a particular degree of excitability. Our aim is to investigate the effect of such heterogeneity level in the strategies used by the network to process and codify neural signals. Our results show that neural heterogeneity has important effects in several properties of interest in neural networks, such as the mean firing rate or the synchronization properties of the network. Then, we find that these effects have strong consequences for two main information processing strategies used in many brain areas: rate coding and temporal coding. More precisely, we show, theoretically and numerically, that certain heterogeneity valueswhich lie within the physiological rage found in actual neurons- optimize the transmission of information in rate and temporal codes in a stochastic resonance fashion, suggesting that cell-to-cell differences are not useful only for population coding, but also for more general coding strategies. Interestingly, we find that networks of heterogeneous neurons tend to behave differently to input changes (such as increments in input rate) depending on the neural coding strategy they are operating in. Such finding suggests the existence of heterogeneity-induced features which are specific for each neural coding strategy.

Dynamics of self-sustained activity in random networks with strong synapses

An understanding of short-term memory requires models of neural networks which are able to sustain activity in the absence of external input for several seconds. About half a century ago, [1] predicted the existence of self-sustained activity in neural networks with strong synapses. Despite this finding, most previous studies on the dynamics of neural networks are restricted to weak synapses, i.e. to a regime where the diffusion approximation is applicable. Recently, it has been shown in simulations that self-sustained activity emerges in networks of integrate-and-fire (IaF) neurons with strong synapses modeled as currents [2] (rather than as conductances, see [3]). Above a critical synaptic weight, the lifetime of self-sustained activity increases rapidly (Fig. ​(Fig.1C).1C). We present a stochastic model of the dynamics of a balanced random network of IaF neurons with current-based synapses in the strong-synapse regime. Based on the network's firing-rate transfer [4], we show that the firing-rate dynamics becomes bistable if the synapses are sufficiently strong: in addition to the quiescent state, a second stable fixed point at moderate firing rates is created (sketched in Fig. ​Fig.1A).1A). Firing-rate fluctuations can destabilize this fixed point, thereby limiting the lifetime of self-sustained activity (sketched in Fig. ​Fig.1B).1B). The magnitude of these fluctuations is mainly determined by the amount of spike-train correlations [5]. Our model explains the existence and the lifetime of self-sustained activity, and how the lifetime depends on the network size, the connectivity, the level of inhibition and the synapse strength. The results of our model are confirmed by network simulations. Figure 1 A. Phase space of the firing-rate dynamics. Dependence of the rate change dr/dt on the rate r (sketch). B. Firing-rate distribution in the self-sustained state (sketch). C. Dependence of the lifetime of self-sustained activity (color coded) on the level ...

Analytical integrate-and-fire neuron models with conductance-based dynamics and realistic PSP time course for event-driven simulation strategies

Recently, we developed models of analytical integrateand-fire (IF) neurons with conductance-based (COBA) dynamics, the gIF neuron models [1]. These models are based on an analytical approximation of the differential equation describing the IF neuron with exponential synaptic conductances and were successfully tested with respect to their response to random and oscillating inputs. Due to their simple mathematical structure, the gIF models are best suited for fast, event-driven simulation strategies. However, the approximations were based on the assumption of instantaneous rise and exponential decay of the postsynaptic potential (PSP). Consequent to this instantaneous rise, spikes were always produced at the onset of the PSP, which might have consequences for the temporal precision of the cellular responses, as the finite rise time of the PSP and the consequent delay between the onset of synaptic input and spike were not present in this model.

Temporal difference learning does not always lead to STDP

Event Abstract Back to Event Temporal difference learning does not always lead to STDP It has been previously shown that a form of temporal difference (TD) learning for predicting the value of the membrane potential of a neuron, at a fixed delay after the neuron received a presynaptic spike, results in a learning rule that is very similar to hebbian STDP (Rao and Sejnowski, 2001). Since this result was obtained using a relatively complex neural model (two-compartmental, with Hodgkin-Huxley-like currents) and a simple setup (a single presynaptic spike followed by a single current pulse), we investigated whether it holds for simpler neural models and more general situations. This is relevant both for the theoretical understanding of this phenomenon and for verifying that it holds in common simulations of spiking neural networks. We studied the same phenomenon using both integrate-and-fire (IAF) and Izhikevich neurons, through simulations and, for the IAF neuron, also analytically. Postsynaptic spikes were generated by single current pulses (as in the original study), by the irregular firing of synaptic afferents, or by a constant input current. For the IAF neuron, we found a hebbian, STDP-like plasticity rule only when postsynaptic spikes were generated by a single current pulse and the reset potential of the neuron was positive. For the same input and negative reset potential, as well as for constant input current (regardless of the reset potential), the resulting plasticity rule was anti-hebbian. For the Izhikevich neuron we obtained hebbian STDP-like plasticity for both constant and pulsed input current. There is a qualitative difference with respect to the IAF case because the Izhikevich neuron incorporates the dynamics of the membrane potential during the onset of the action potential. By adding an action potential of non-zero duration to the IAF model, the shape of the plasticity function changes significantly and becomes similar to hebbian STDP. This shows that the plasticity function resulted from TD learning depends critically on whether the neuron adapts its synapses to learn the shape of its action potential or not. However, the shape of the action potential is commonly considered not to carry information. When we consider just the TD learning of the sub-threshold dynamics of the membrane potential, the shape of the resulted learning function can loose its similarity with hebbian STDP. For both neural models, in the case of irregular synaptic input there was no clear relationship between the plastic changes predicted by TD learning and the temporal delay between the pre- and postsynaptic spikes. Moreover, the sign of these plastic changes did not depend uniquely on the sign of the temporal delay. In conclusion, TD learning in spiking neurons does not always lead to hebbian STDP.

Theory of neuronal spike densities for synchronous activity in cortical feed-forward networks

Synchronization of spiking activity in neuronal networks of the cortex has been proposed as a mechanism underlying higher brain functions. This idea is challenged by ongoing cortical activity generating large fluctuations in synaptic input, causing the neurons to operate in a noisy environment. The propagation of synchronized spiking in feed-forward subnetworks (synfire chains) has been studied to demonstrate the feasibility of precise spike timing [1]. However, the theoretical analysis of even this toy model is impeded by the intricacy of calculating the distribution of spike times in response to time-dependent inputs. Therefore, most quantitative results rely on simulations or semi-numerical methods, and insight into the structure of the dynamics is limited. Recently [2], we showed that for a biophysically plausible integrate-andfire neuron model the probability of emitting a response spike is concentrated on the rising phase of the membrane potential transient caused by synchronous input and that during this time the instantaneous spiking rate is governed by the derivative of this transient. These observations were confirmed by an ad hoc calculation of the neuron's spike density. A corresponding instantaneous rate model enabled us to investigate the synchronization dynamics analytically. Inherent to the approach, the theory for the spike density breaks down at the peak of the membrane potential transient and during its descent. Meanwhile based on an exact series expression for the first passage time density of differentiable random processes (Wiener-Rice series) new successful approximations have been presented [3]. Here, we use the first term of the series to approximate the spike density of a stochastic integrateand-fire neuron model receiving time-dependent input.

Mesoscopic model of balanced neuron networks using a Master equation formalism

Cortical activity in awake animals manifests highly complex behaviour [1]. It is during this regime that the main computational tasks are performed and no model is yet able to explain how this complex dynamics is exploited to provide a fast and accurate information processing. However, many efforts have been devoted to the study of how such activity emerges. Balanced networks have been introduced as a possible model to generate dynamical states similar to the biological ones [2]. The stability of such states was studied for current-based Integrate-And-Fire (IAF) neurons with respect to external input and excitatory-inhibitory synaptic strength ratio [3]. In particular, stable asynchronous irregular (AI) states with a relatively low level of activity have been obtained. Recently, AI states have been observed in balanced networks of conductance-based IAF neurons with self-sustained activity [4]. However, no simple description of the network activity dynamics has been developed yet. First-order mean-field approximation fails to describe these networks because of their inherent dynamics which rely dramatically on activity fluctuations. Moreover, the thermodynamic limit is usually performed for randomly connected networks despite the lack of biological relevance. We introduce here a new framework in which network dynamics as well as inherent neuron behaviour is taken into account. We aim to obtain a reduced description of mesoscopic balanced networks where finite size effects are not neglected. The model is intended to describe AI states far from critical boundaries where long-term behaviours appear. Furthermore, we set the spatial and temporal scales of the model by using biological data. Using the master equation formalism, we derive a second-order mean-field set of ordinary differential equations. The transition matrix necessary in the master equation context is computed based on the Fokker-Plank approach. Conductance-based as well as current-based IAF neurons are constructed. The kernel of this formalism lays in the way activity micro-fluctuations are modelled. We discuss different possibilities and considerations in regard to this question. This model provides at the same time an extracellular and a sub-threshold description of finite size neuron networks. Once the couplings will be adjusted, it will be possible to build a large-scale model of cortical area with specific architectures, where the fundamental unit is the randomly connected network. We further discuss the possibility to compare large-scale behaviour observed in voltage-sensitive dyes experiments with our model.

Spiking neuron devices consisting of single-flux-quantum circuits

Single-flux-quantum (SFQ) circuits can be used for making spiking neuron devices, which are useful elements for constructing intelligent, brain-like computers. The device we propose is based on the leaky integrate-and-fire neuron (IFN) model and uses a SFQ pulse as an action signal or a spike of neurons. The operation of the neuron device is confirmed by computer simulator. It can operate with a short delay of 100ps or less and is the highest-speed neuron device ever reported.

Time-series prediction with single integrate-and-fire neuron

In this paper, a learning algorithm for a single integrate-and-fire neuron (IFN) is proposed and tested for various applications in which a multilayer perceptron neural network is conventionally used. It is found that a single IFN is sufficient for the applications that require a number of neurons in different hidden layers of a conventional neural network. Several benchmark and real-life problems of classification and time-series prediction have been illustrated. It is observed that the inclusion of some more biological phenomenon in an artificial neural network can make it more powerful.

Dynamic stochastic synapses as computational units.

In most neural network models, synapses are treated as static weights that change only with the slow time scales of learning. It is well known, however, that synapses are highly dynamic and show use-dependent plasticity over a wide range of time scales. Moreover, synaptic transmission is an inherently stochastic process: a spike arriving at a presynaptic terminal triggers the release of a vesicle of neurotransmitter from a release site with a probability that can be much less than one. We consider a simple model for dynamic stochastic synapses that can easily be integrated into common models for networks of integrate-and-fire neurons (spiking neurons). The parameters of this model have direct interpretations in terms of synaptic physiology. We investigate the consequences of the model for computing with individual spikes and demonstrate through rigorous theoretical results that the computational power of the network is increased through the use of dynamic synapses.