Leaky Integrate-and-fire Neuron Research Articles

Learning optimisation by high firing irregularity

In a network of leaky integrate-and-fire (LIF) neurons, we investigate the functional role of irregular spiking at high rates. Irregular spiking is produced by either employing the partial somatic reset mechanism on every LIF neuron of the network or by using temporally correlated inputs. In both the benchmark problem of XOR (exclusive-OR) and in a general-sum game, it is shown that irrespective of the mechanism that is used to produce it, high firing irregularity enhances the learning capability of the spiking neural network trained with reward-modulated spike-timing-dependent plasticity. These results suggest that the brain may be utilising high firing irregularity for the purposes of learning optimisation.

Inferring interactions in assemblies of stochastic integrate-and-fire neurons from spike recordings: method, applications and software

Multi-electrode recordings make possible the simultaneous spiking activity of tens of neurons for hours. An important issue is to reconstruct the network of functional interactions between the cells from the observed correlated activity. We have recently designed a fast inference method for this purpose, in which the cells are modeled as Leaky Integrate-and-Fire (LIF) neurons, coupled through a set of interactions Jij [1,2]. Each LIF neuron i receives a stochastic current, equal to the sum of an external current Ii and of a Gaussian white noise process with variance s2. The LIF model implicitly defines the likelihood P of the spiking times {tj,k} given the currents Ii and synaptic interactions Jij. Given the spiking times {tj,k} we infer the couplings and currents by maximizing P (a priori information can be considered, too, see below). Though P can be, in principle, calculated through the numerical resolution of the Fokker-Planck equation associated to the LIF dynamical equations [3], this approach is too slow to treat data sets with tens of neurons and hundreds of thousands of spikes in a reasonable time. In our approach we approximate P from the contribution coming from the most probable trajectory for the potential for each cell i, referred to as Vi*(t). This approximation is exact when the amplitude s of the noise is small. The determination of Vi*(t) was done numerically by Paninski for one cell in [4]. We have found an exact procedure to determine Vi*(t) analytically in a time growing linearly with the number of spikes and quadratically with the number of neurons, which allows us to process very large recordings [2]. Once the most probable trajectory for the potential has been determined for a set of interactions and currents, we calculate the corresponding log-likelihood and maximize it over the interactions and the currents using convex optimizations methods. Our algorithm has been tested again artificially generated data (with up to 160 cells, 20 millions spikes), and real experimental data (recordings of 32 to 60 ganglion cells in the salamander retina, data courtesy of M. Meister) [1,2]. As an example, it takes us about 30 seconds to process a set of 120,000 spikes fired by 32 cells on a commercial desktop computer. We present a software package, running in C, Matlab and Mathematica, offering a practical implementation of our inference method. The program requires as an input the list of spiking times and cell numbers. The user can set various parameters of the model or choose to infer them from the data. A choice of priors over the interactions (based on the L1 and L2 norms) are proposed. Various optimization routines, making use or not of the second derivatives of the log-likelihood are available. The output of the program is the matrix of interactions Jij, with the currents Ii on the diagonal, and the error bars (statistical uncertainties) on those most likely values.

Dynamics of Networks of Excitatory and Inhibitory Neurons in Response to Time-Dependent Inputs

We investigate the dynamics of recurrent networks of excitatory (E) and inhibitory (I) neurons in the presence of time-dependent inputs. The dynamics is characterized by the network dynamical transfer function, i.e., how the population firing rate is modulated by sinusoidal inputs at arbitrary frequencies. Two types of networks are studied and compared: (i) a Wilson–Cowan type firing rate model; and (ii) a fully connected network of leaky integrate-and-fire (LIF) neurons, in a strong noise regime. We first characterize the region of stability of the “asynchronous state” (a state in which population activity is constant in time when external inputs are constant) in the space of parameters characterizing the connectivity of the network. We then systematically characterize the qualitative behaviors of the dynamical transfer function, as a function of the connectivity. We find that the transfer function can be either low-pass, or with a single or double resonance, depending on the connection strengths and synaptic time constants. Resonances appear when the system is close to Hopf bifurcations, that can be induced by two separate mechanisms: the I–I connectivity and the E–I connectivity. Double resonances can appear when excitatory delays are larger than inhibitory delays, due to the fact that two distinct instabilities exist with a finite gap between the corresponding frequencies. In networks of LIF neurons, changes in external inputs and external noise are shown to be able to change qualitatively the network transfer function. Firing rate models are shown to exhibit the same diversity of transfer functions as the LIF network, provided delays are present. They can also exhibit input-dependent changes of the transfer function, provided a suitable static non-linearity is incorporated.

Research on Virtual-Switch Characteristics of Noise-Enhanced Stochastic Resonance Using Leaky Integrate-and-Fire Model

Virtual-switch characteristics of noise-induced stochastic resonance using leaky integrate-and-fire (LIF) model was focused in this article. For LIF neuron model, Gaussian white noise was added to the simulating system. The condition of spike-starting was studied through the system. With the increase of noise intensity, the stimulated spikes became more and more intensive. With proper white noise intensity, the amount of excitatory spikes was enough to generate spike trains. We expected to design a virtual-switch utilizing this mechanism for neural data processing field.

Maximum Likelihood Decoding of Neuronal Inputs from an Interspike Interval Distribution

An expression for the probability distribution of the interspike interval of a leaky integrate-and-fire (LIF) model neuron is rigorously derived, based on recent theoretical developments in the theory of stochastic processes. This enables us to find for the first time a way of developing maximum likelihood estimates (MLE) of the input information (e.g., afferent rate and variance) for an LIF neuron from a set of recorded spike trains. Dynamic inputs to pools of LIF neurons both with and without interactions are efficiently and reliably decoded by applying the MLE, even within time windows as short as 25 msec.

Comparative study of different integrate-and-fire neurons: Spontaneous activity, dynamical response, and stimulus-induced correlation

Stochastic integrate-and-fire (IF) neuron models have found widespread applications in computational neuroscience. Here we present results on the white-noise-driven perfect, leaky, and quadratic IF models, focusing on the spectral statistics (power spectra, cross spectra, and coherence functions) in different dynamical regimes (noise-induced and tonic firing regimes with low or moderate noise). We make the models comparable by tuning parameters such that the mean value and the coefficient of variation of the interspike interval (ISI) match for all of them. We find that, under these conditions, the power spectrum under white-noise stimulation is often very similar while the response characteristics, described by the cross spectrum between a fraction of the input noise and the output spike train, can differ drastically. We also investigate how the spike trains of two neurons of the same kind (e.g., two leaky IF neurons) correlate if they share a common noise input. We show that, depending on the dynamical regime, either two quadratic IF models or two leaky IFs are more strongly correlated. Our results suggest that, when choosing among simple IF models for network simulations, the details of the model have a strong effect on correlation and regularity of the output.

Discrimination with Spike Times and ISI Distributions

We study the discrimination capability of spike time sequences using the Chernoff distance as a metric. We assume that spike sequences are generated by renewal processes and study how the Chernoff distance depends on the shape of interspike interval (ISI) distribution. First, we consider a lower bound to the Chernoff distance because it has a simple closed form. Then we consider specific models of ISI distributions such as the gamma, inverse gaussian (IG), exponential with refractory period (ER), and that of the leaky integrate-and-fire (LIF) neuron. We found that the discrimination capability of spike times strongly depends on high-order moments of ISI and that it is higher when the spike time sequence has a larger skewness and a smaller kurtosis. High variability in terms of coefficient of variation (CV) does not necessarily mean that the spike times have less discrimination capability. Spike sequences generated by the gamma distribution have the minimum discrimination capability for a given mean and variance of ISI. We used series expansions to calculate the mean and variance of ISIs for LIF neurons as a function of the mean input level and the input noise variance. Spike sequences from an LIF neuron are more capable of discrimination than those of IG and gamma distributions when the stationary voltage level is close to the neuron's threshold value of the neuron.

MIIND (Multiple Interacting Instantiations of Neural Dynamics) - bridging neural dynamics and cognitive modelling

Event Abstract Back to Event MIIND (Multiple Interacting Instantiations of Neural Dynamics) - bridging neural dynamics and cognitive modelling Marc De Kamps1* and Frank Van Der Velde2 1 University of Leeds, United Kingdom 2 University of Leiden, Netherlands MIIND is a C++ framework with a PYTHON interface which facilitates modelling in cognitive neuroscience. In MIIND one can create networks in terms of spatial configurations. The node of such networks can be endowed with algorithms that evolve the state of the nodes. In this way network processes can be modelled in a very generic way. In principle any coupled system of equations which can be represented by a sparse irregular network can be modelled. Users can provide their own algorithms in a transparent way. MIIND also has some predefined algorithms: Wilson-Cowan dynamics and population density techniques for leaky-integrate-and-fire (LIF) neurons. Wilson-Cowan dynamics is widely used in models of high level cognition. Population density techniques describe large populations of spiking neurons in terms of a density function which captures the characteristics of the population in a way that is often more efficient than a straightforward simulation of the population. Networks of such equations are closely linked to coupled systems of Fokker-Planck equations. Since MIIND can create complex spatial structures and provides algorithms for modelling neural dynamics, it is suitable for modelling high level cognition. One example of its application is modelling object-based attention in vision in terms of interacting hierarchical neural networks, the so-called neural blackboard architecture. Another application is the investigation of the behaviour of the dynamics of cortical circuits. MIIND does not provide simulations of spiking neurons, such as NEST, NEURON, GENESIS or MOOSE, although it could encapsulate algorithms which drive such simulations. MIIND does not provide a monolithic interface, but offers modelling support at various levels. Modellers may decide, for example, to use only the representation of sparse networks, or to use the simulation loop for other than neural processes. Developers can easily borrow parts of MIIND while disregarding the parts that are not of interest. MIIND also comes with a library that can implement filter operations between layers of nodes. In this way it is easy to implement large filter bank structures, which are used in biologically inspired models of object recognition, such as HMAX and successors. This opens the possibility to model biological vision on the one hand, and to apply the principles to biologically inspired artificial vision on the other hand. MIIND uses ROOT for visualisation and data management. ROOT is a powerful Open Source package, developed by CERN. It relies on the GNU Scientific Library (GSL) for scientific computing. MIIND is still in its infancy. Important future challenges are: the development of parallelisation of the simulation loop, extending the visualisation capabilities and the development of population density techniques which go beyond LIF neurons. Conference: Neuroinformatics 2008, Stockholm, Sweden, 7 Sep - 9 Sep, 2008. Presentation Type: Poster Presentation Topic: Large Scale Modeling Citation: De Kamps M and Van Der Velde F (2008). MIIND (Multiple Interacting Instantiations of Neural Dynamics) - bridging neural dynamics and cognitive modelling. Front. Neuroinform. Conference Abstract: Neuroinformatics 2008. doi: 10.3389/conf.neuro.11.2008.01.090 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 28 Jul 2008; Published Online: 28 Jul 2008. * Correspondence: Marc De Kamps, University of Leeds, Leeds, United Kingdom, dekamps@comp.leeds.ac.uk Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Marc De Kamps Frank Van Der Velde Google Marc De Kamps Frank Van Der Velde Google Scholar Marc De Kamps Frank Van Der Velde PubMed Marc De Kamps Frank Van Der Velde Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Temporal coding of continuously-varying inputs

In many neural circuits, precise patterns of spike timing contain information beyond that contained in mean firing rates. Here we illustrate a simple mechanism by which an ensemble of leaky-integrate-and-fire (LIF) neurons can represent continuously-varying input signals in a timing code. Neurons that are post-synaptic to this ensemble can reliably extract these signals (or functions thereof) in the absence of both spike time coincidence and firing rate variations. Irregular firing is often modelled phenomenologically, for example as a Poisson process with a rate that depends on synaptic input. In contrast, the irregular firing of our LIF neurons is a deterministic consequence of wide variations in applied current over the space of inputs (e.g. Figure ​Figure1A).1A). Applied current functions of this kind can arise from weighted output from a previous layer, and we discuss their establishment via Hebbian plasticity. By inclining these functions along a preferred direction, and scaling the peaks, we obtain a continuum between timing and rate codes. Figure 1 Temporal coding and decoding with LIF neurons. A, Net synaptic current (arbitrary units) experienced by an example LIF neuron, as a function of two inputs (X and Y). B, Irregular firing in 50 different neurons (each with different current functions) as ...

BvP Neurons Exhibit a Larger Variety in Statistics of Inter-Spike Intervals than LIF Neurons

It was recently found that the leaky integrate-and-fire (LIF) model with the assumption of temporally uncorrelated inputs cannot account for the spiking characteristics of in vivo cortical neurons. Specifically, the inter-spike interval (ISI) distributions of some cortical neurons are known to exhibit relatively large skewness to variation, whereas the LIF model cannot realize such statistics with any combination of model parameters. In the present paper, we show that the Bonhoeffer–van del Pol (BvP) model incorporating the same assumption of uncorrelated inputs can, by contrast, exhibit large skewness values. In this case, the large values of the skewness coefficient are caused by the mixture of widely distributed ISIs and short-and-constant ISIs induced by a sub-threshold oscillation peculiar to Class II neurons, such as the BvP neuron.

Analytical Integrate-and-Fire Neuron Models with Conductance-Based Dynamics for Event-Driven Simulation Strategies

Event-driven simulation strategies were proposed recently to simulate integrate-and-fire (IF) type neuronal models. These strategies can lead to computationally efficient algorithms for simulating large-scale networks of neurons; most important, such approaches are more precise than traditional clock-driven numerical integration approaches because the timing of spikes is treated exactly. The drawback of such event-driven methods is that in order to be efficient, the membrane equations must be solvable analytically, or at least provide simple analytic approximations for the state variables describing the system. This requirement prevents, in general, the use of conductance-based synaptic interactions within the framework of event-driven simulations and, thus, the investigation of network paradigms where synaptic conductances are important. We propose here a number of extensions of the classical leaky IF neuron model involving approximations of the membrane equation with conductance-based synaptic current, which lead to simple analytic expressions for the membrane state, and therefore can be used in the event-driven framework. These conductance-based IF (gIF) models are compared to commonly used models, such as the leaky IF model or biophysical models in which conductances are explicitly integrated. All models are compared with respect to various spiking response properties in the presence of synaptic activity, such as the spontaneous discharge statistics, the temporal precision in resolving synaptic inputs, and gain modulation under in vivo-like synaptic bombardment. Being based on the passive membrane equation with fixed-threshold spike generation, the proposed gIF models are situated in between leaky IF and biophysical models but are much closer to the latter with respect to their dynamic behavior and response characteristics, while still being nearly as computationally efficient as simple IF neuron models. gIF models should therefore provide a useful tool for efficient and precise simulation of large-scale neuronal networks with realistic, conductance-based synaptic interactions.

Cortical network modeling: Analytical methods for firing rates and some properties of networks of LIF neurons

The circuitry of cortical networks involves interacting populations of excitatory (E) and inhibitory (I) neurons whose relationships are now known to a large extent. Inputs to E- and I-cells may have their origins in remote or local cortical areas. We consider a rudimentary model involving E- and I-cells. One of our goals is to test an analytic approach to finding firing rates in neural networks without using a diffusion approximation and to this end we consider in detail networks of excitatory neurons with leaky integrate-and-fire (LIF) dynamics. A simple measure of synchronization, denoted by S q , where q is between 0 and 100 is introduced. Fully connected E-networks have a large tendency to become dominated by synchronously firing groups of cells, except when inputs are relatively weak. We observed random or asynchronous firing in such networks with diverse sets of parameter values. When such firing patterns were found, the analytical approach was often able to accurately predict average neuronal firing rates. We also considered several properties of E–E networks, distinguishing several kinds of firing pattern. Included were those with silences before or after periods of intense activity or with periodic synchronization. We investigated the occurrence of synchronized firing with respect to changes in the internal excitatory postsynaptic potential (EPSP) magnitude in a network of 100 neurons with fixed values of the remaining parameters. When the internal EPSP size was less than a certain value, synchronization was absent. The amount of synchronization then increased slowly as the EPSP amplitude increased until at a particular EPSP size the amount of synchronization abruptly increased, with S 5 attaining the maximum value of 100%. We also found network frequency transfer characteristics for various network sizes and found a linear dependence of firing frequency over wide ranges of the external afferent frequency, with non-linear effects at lower input frequencies. The theory may also be applied to sparsely connected networks, whose firing behaviour was found to change abruptly as the probability of a connection passed through a critical value. The analytical method was also found to be useful for a feed-forward excitatory network and a network of excitatory and inhibitory neurons.

A Neuromorphic Depth-From-Motion Vision Model With STDP Adaptation

We propose a simplified depth-from-motion vision model based on leaky integrate-and-fire (LIF) neurons for edge detection and two-dimensional depth recovery. In the model, every LIF neuron is able to detect the irradiance edges passing through its receptive field in an optical flow field, and respond to the detection by firing a spike when the neuron's firing criterion is satisfied. If a neuron fires a spike, the time-of-travel of the spike-associated edge is transferred as the prediction information to the next synapse-linked neuron to determine its state. Correlations between input spikes and their timing thus encode depth in the visual field. The adaptation of synapses mediated by spike-timing-dependent plasticity is used to improve the algorithm's robustness against inaccuracy caused by spurious edge propagation. The algorithm is characterized on both artificial and real image sequences. The implementation of the algorithm in analog very large scale integrated (aVLSI) circuitry is also discussed.

Precision Constrained Stochastic Resonance in a Feedforward Neural Network

Stochastic resonance (SR) is a phenomenon in which the response of a nonlinear system to a subthreshold information-bearing signal is optimized by the presence of noise. By considering a nonlinear system (network of leaky integrate-and-fire (LIF) neurons) that captures the functional dynamics of neuronal firing, we demonstrate that sensory neurons could, in principle harness SR to optimize the detection and transmission of weak stimuli. We have previously characterized this effect by use of signal-to-noise ratio (SNR). Here in addition to SNR, we apply an entropy-based measure (Fisher information) and compare the two measures of quantifying SR. We also discuss the performance of these two SR measures in a full precision floating point model simulated in Java and in a precision limited integer model simulated on a field programmable gate array (FPGA). We report in this study that stochastic resonance which is mainly associated with floating point implementations is possible in both a single LIF neuron and a network of LIF neurons implemented on lower resolution integer based digital hardware. We also report that such a network can improve the SNR and Fisher information of the output over a single LIF neuron.

Interspike interval correlations, memory, adaptation, and refractoriness in a leaky integrate-and-fire model with threshold fatigue.

Neuronal adaptation as well as interdischarge interval correlations have been shown to be functionally important properties of physiological neurons. We explore the dynamics of a modified leaky integrate-and-fire (LIF) neuron, referred to as the LIF with threshold fatigue, and show that it reproduces these properties. In this model, the postdischarge threshold reset depends on the preceding sequence of discharge times. We show that in response to various classes of stimuli, namely, constant currents, step currents, white gaussian noise, and sinusoidal currents, the model exhibits new behavior compared with the standard LIF neuron. More precisely, (1) step currents lead to adaptation, that is, a progressive decrease of the discharge rate following the stimulus onset, while in the standard LIF, no such patterns are possible; (2) a saturation in the firing rate occurs in certain regimes, a behavior not seen in the LIF neuron; (3) interspike intervals of the noise-driven modified LIF under constant current are correlated in a way reminiscent of experimental observations, while those of the standard LIF are independent of one another; (4) the magnitude of the correlation coefficients decreases as a function of noise intensity; and (5) the dynamics of the sinusoidally forced modified LIF are described by iterates of an annulus map, an extension to the circle map dynamics displayed by the LIF model. Under certain conditions, this map can give rise to sensitivity to initial conditions and thus chaotic behavior.

Burst firing improves the detection of weak signals in spike trains

The detection of weak sensory signals is an important aspect of neuronal information processing. Behaviorally relevant signals are often encoded as perturbations of on-going spiking activity in primary afferents. Here, we show that a biologically plausible model, the leaky integrate-and-fire (LIAF) neuron, is capable of efficient and reliable detection of a single spike added to baseline activity. Detection performance is dependent on the statistical properties of the spike train. For the type of statistics considered here, an LIAF neuron can distinguish between a correct detection by means of burst firing, whereas false alarms tend to result in isolated spikes. The methods are illustrated by an application to electrosensory afferents of weakly electric fish.

Computing with the leaky integrate-and-fire neuron: logarithmic computation and multiplication.

The leaky integrate-and-fire (LIF) model of neuronal spiking (Stein 1967) provides an analytically tractable formalism of neuronal firing rate in terms of a neuron's membrane time constant, threshold, and refractory period. LIF neurons have mainly been used to model physiologically realistic spike trains, but little application of the LIF model appears to have been made in explicitly computational contexts. In this article, we show that the transfer function of a LIF neuron provides, over a wide-parameter range, a compressive nonlinearity sufficiently close to that of the logarithm so that LIF neurons can be used to multiply neural signals by mere addition of their outputs yielding the logarithm of the product. A simulation of the LIF multiplier shows that under a wide choice of parameters, a LIF neuron can log-multiply its inputs to within a 5% relative error.

Summation and multiplication: two distinct operation domains of leaky integrate-and-fire neurons

The spiking frequency of a leaky integrate-and-fire (LIF) neuron can be proportional to the sum or the product of a number n≥2 of input frequencies. In the paper, the parameter domains (discharge time constants and synaptic weights) for these two operation modes are defined theoretically and studied by simulations. Summation is based on the frequency division principle and requires discharge time constant as long as possible. In this mode, the LIF neuron is subject to phase locking effects and is insensitive to the irregularity of the input spike trains. Multiplication is based on coincidence detection and requires shorter time constants. Simulations show that the quality of the mulutiplication function decreases for large irregularities of the input spike trains, that there is an optimum value of the synaptic weights and that there is a consequent input-output frequency level drop. In the brain, the frequency level decrease observed from retina to higher cortical areas might indicate the presence of mult...