Axonal Delays Research Articles

A simple programmable axonal delay scheme for spiking neural networks

The paper addresses the need for a compact axonal delay circuit for analogue hardware neural networks. The delay is implemented using the charging of a capacitive node via a pMOSFET operating in sub-threshold. The duration of the delay can be programmed by adjusting the associated MOSFET gate voltage and delays spanning five orders of magnitude, between microseconds and tens of milliseconds are shown experimentally. A model is reported that allows for prediction of the delay. This analysis is supported by experimental results from circuits fabricated in AMS 0.35μm CMOS. An approach for integration of the axon circuit into a neuron circuit is also considered, where both single and burst spikes are possible.

Synthesis of neural networks for spatio-temporal spike pattern recognition and processing

The advent of large scale neural computational platforms has highlighted the lack of algorithms for synthesis of neural structures to perform predefined cognitive tasks. The Neural Engineering Framework (NEF) offers one such synthesis, but it is most effective for a spike rate representation of neural information, and it requires a large number of neurons to implement simple functions. We describe a neural network synthesis method that generates synaptic connectivity for neurons which process time-encoded neural signals, and which makes very sparse use of neurons. The method allows the user to specify—arbitrarily—neuronal characteristics such as axonal and dendritic delays, and synaptic transfer functions, and then solves for the optimal input-output relationship using computed dendritic weights. The method may be used for batch or online learning and has an extremely fast optimization process. We demonstrate its use in generating a network to recognize speech which is sparsely encoded as spike times.

Reciprocal inhibition and slow calcium decay in perigeniculate interneurons explain changes of spontaneous firing of thalamic cells caused by cortical inactivation

The role of cortical feedback in the thalamocortical processing loop has been extensively investigated over the last decades. With an exception of several cases, these searches focused on the cortical feedback exerted onto thalamo-cortical relay (TC) cells of the dorsal lateral geniculate nucleus (LGN). In a previous, physiological study, we showed in the cat visual system that cessation of cortical input, despite decrease of spontaneous activity of TC cells, increased spontaneous firing of their recurrent inhibitory interneurons located in the perigeniculate nucleus (PGN). To identify mechanisms underlying such functional changes we conducted a modeling study in NEURON on several networks of point neurons with varied model parameters, such as membrane properties, synaptic weights and axonal delays. We considered six network topologies of the retino-geniculo-cortical system. All models were robust against changes of axonal delays except for the delay between the LGN feed-forward interneuron and the TC cell. The best representation of physiological results was obtained with models containing reciprocally connected PGN cells driven by the cortex and with relatively slow decay of intracellular calcium. This strongly indicates that the thalamic reticular nucleus plays an essential role in the cortical influence over thalamo-cortical relay cells while the thalamic feed-forward interneurons are not essential in this process. Further, we suggest that the dependence of the activity of PGN cells on the rate of calcium removal can be one of the key factors determining individual cell response to elimination of cortical input.Electronic supplementary materialThe online version of this article (doi:10.1007/s10827-012-0430-8) contains supplementary material, which is available to authorized users.

Signal-to-noise ratio in the membrane potential of the owl's auditory coincidence detectors.

Owls use interaural time differences (ITDs) to locate a sound source. They compute ITD in a specialized neural circuit that consists of axonal delay lines from the cochlear nucleus magnocellularis (NM) and coincidence detectors in the nucleus laminaris (NL). Recent physiological recordings have shown that tonal stimuli induce oscillatory membrane potentials in NL neurons (Funabiki K, Ashida G, Konishi M. J Neurosci 31: 15245-15256, 2011). The amplitude of these oscillations varies with ITD and is strongly correlated to the firing rate. The oscillation, termed the sound analog potential, has the same frequency as the stimulus tone and is presumed to originate from phase-locked synaptic inputs from NM fibers. To investigate how these oscillatory membrane potentials are generated, we applied recently developed signal-to-noise ratio (SNR) analysis techniques (Kuokkanen PT, Wagner H, Ashida G, Carr CE, Kempter R. J Neurophysiol 104: 2274-2290, 2010) to the intracellular waveforms obtained in vivo. Our theoretical prediction of the band-limited SNRs agreed with experimental data for mid- to high-frequency (>2 kHz) NL neurons. For low-frequency (≤2 kHz) NL neurons, however, measured SNRs were lower than theoretical predictions. These results suggest that the number of independent NM fibers converging onto each NL neuron and/or the population-averaged degree of phase-locking of the NM fibers could be significantly smaller in the low-frequency NL region than estimated for higher best-frequency NL.

Inferior olive mirrors joint dynamics to implement an inverse controller

To produce smooth and coordinated motion, our nervous systems need to generate precisely timed muscle activation patterns that, due to axonal conduction delay, must be generated in a predictive and feedforward manner. Kawato proposed that the cerebellum accomplishes this by acting as an inverse controller that modulates descending motor commands to predictively drive the spinal cord such that the musculoskeletal dynamics are canceled out. This and other cerebellar theories do not, however, account for the rich biophysical properties expressed by the olivocerebellar complex's various cell types, making these theories difficult to verify experimentally. Here we propose that a multizonal microcomplex's (MZMC) inferior olivary neurons use their subthreshold oscillations to mirror a musculoskeletal joint's underdamped dynamics, thereby achieving inverse control. We used control theory to map a joint's inverse model onto an MZMC's biophysics, and we used biophysical modeling to confirm that inferior olivary neurons can express the dynamics required to mirror biomechanical joints. We then combined both techniques to predict how experimentally injecting current into the inferior olive would affect overall motor output performance. We found that this experimental manipulation unmasked a joint's natural dynamics, as observed by motor output ringing at the joint's natural frequency, with amplitude proportional to the amount of current. These results support the proposal that the cerebellum-in particular an MZMC-is an inverse controller; the results also provide a biophysical implementation for this controller and allow one to make an experimentally testable prediction.

STDP encodes oscillation frequencies in the connections of recurrent networks of spiking neurons

Spike-timing-dependent plasticity (STDP) is a learning rule that updates synaptic strengths based on the relative timing of pre- and post-synaptic spikes. Unlike rate-based Hebbian learning, STDP can potentially encode fast temporal correlations in neuronal activity, such as oscillations, in the functional structure of networks of neurons that have axonal and dendritic propagation delays. The motivation behind this study was to understand the different ways that spatiotemporal patterns can be learnt by the recurrent connections in a network of neurons with STDP present. This understanding is vital to uncovering the mechanisms by which basic learning and information processing tasks are performed throughout the brain. A specific example in which these mechanisms may contribute is in explaining how the brain can perceive the pitch of complex sounds up to 300Hz. This work employs and builds upon the analytical framework for learning with STDP used in a previous study [1]. In this study, the changes made by additive STDP to synaptic strengths in recurrent networks with axonal delays receiving oscillatory inputs were investigated analytically with the Poisson neuron model and verified through simulations with leaky integrate-and-fire (LIF) neurons. Frequencies between 100-300Hz were considered, which correspond to the modulation frequencies found in the auditory brainstem representing the fundamental frequency of different natural sounds. The analysis and simulations found that connections were selectively potentiated and depressed based on their axonal delay in such a way that the delays of the strong connections in the network “resonated” with the input frequency (Figure ​(Figure1A).1A). The trained networks were found to respond selectively to the frequency they were trained with (Figure ​(Figure1B).1B). Higher frequencies (e.g. 240Hz as shown here) would always be learnt by the network, but in order to show a selective response after learning they needed faster neuronal and synaptic time constants (see details in Figure ​Figure11 caption). Figure 1 Results of simulations with networks of 10,000 LIF neurons. A: The mean synaptic weight with axonal delay of a network before learning (dashed: all weights initially set to 0.0025 with an even spread of delays over 1-10ms) and after learning for 20,000s ...

Impaired long-range synchronization of gamma oscillations in the neocortex of a mouse lacking Kv3.2 potassium channels

Inhibitory interneurons play a critical role in the generation of gamma (20-50 Hz) oscillations, either by forming mutually inhibitory networks or as part of recurrent networks with pyramidal cells. A key property of fast spiking interneurons is their ability to generate brief spikes and high-frequency spike trains with little accommodation. However, the role of their firing properties in network oscillations has not been tested in vivo. Studies in hippocampus in vitro have shown that high-frequency spike doublets in interneurons play a key role in the long-range synchronization of gamma oscillations with little phase lag despite long axonal conduction delays. We generated a knockout (KO) mouse lacking Kv3.2 potassium channel subunits, where infragranular inhibitory interneurons lose the ability both to sustain high-frequency firing and reliably generate high-frequency spike doublets. We recorded cortical local field potentials in anesthetized and awake, restrained mice. Spontaneous activity of the KO and the wild-type (WT) showed similar content of gamma and slow (0.1-15 Hz) frequencies, but the KO showed a significantly larger decay of synchronization of gamma oscillations with distance. Coronal cuts in the cortex of WT mice decreased synchronization to values similar to the intact KO. The synchronization of the slow oscillation showed little decay with distance in both mice and was largely reduced after coronal cuts. Our results show that the firing properties of inhibitory interneurons are critical for long-range synchronization of gamma oscillations, and emphasize that intrinsic electrophysiological properties of single cells may play a key role in the spatiotemporal characteristics of network activity.

Reciprocal inhibition and slow calcium decay in perigeniculate interneurons explain changes of spontaneous firing of thalamic cells caused by cortical inactivation

Reciprocal inhibition and slow calcium decay in perigeniculate interneurons explain changes of spontaneous firing of thalamic cells caused by cortical inactivation

Emergent Auditory Feature Tuning in a Real-Time Neuromorphic VLSI System

Many sounds of ecological importance, such as communication calls, are characterized by time-varying spectra. However, most neuromorphic auditory models to date have focused on distinguishing mainly static patterns, under the assumption that dynamic patterns can be learned as sequences of static ones. In contrast, the emergence of dynamic feature sensitivity through exposure to formative stimuli has been recently modeled in a network of spiking neurons based on the thalamo-cortical architecture. The proposed network models the effect of lateral and recurrent connections between cortical layers, distance-dependent axonal transmission delays, and learning in the form of Spike Timing Dependent Plasticity (STDP), which effects stimulus-driven changes in the pattern of network connectivity. In this paper we demonstrate how these principles can be efficiently implemented in neuromorphic hardware. In doing so we address two principle problems in the design of neuromorphic systems: real-time event-based asynchronous communication in multi-chip systems, and the realization in hybrid analog/digital VLSI technology of neural computational principles that we propose underlie plasticity in neural processing of dynamic stimuli. The result is a hardware neural network that learns in real-time and shows preferential responses, after exposure, to stimuli exhibiting particular spectro-temporal patterns. The availability of hardware on which the model can be implemented, makes this a significant step toward the development of adaptive, neurobiologically plausible, spike-based, artificial sensory systems.

Axonal conduction delays

Axonal conduction delays

Does Spike-Timing-Dependent Synaptic Plasticity Couple or Decouple Neurons Firing in Synchrony?

Spike synchronization is thought to have a constructive role for feature integration, attention, associative learning, and the formation of bidirectionally connected Hebbian cell assemblies. By contrast, theoretical studies on spike-timing-dependent plasticity (STDP) report an inherently decoupling influence of spike synchronization on synaptic connections of coactivated neurons. For example, bidirectional synaptic connections as found in cortical areas could be reproduced only by assuming realistic models of STDP and rate coding. We resolve this conflict by theoretical analysis and simulation of various simple and realistic STDP models that provide a more complete characterization of conditions when STDP leads to either coupling or decoupling of neurons firing in synchrony. In particular, we show that STDP consistently couples synchronized neurons if key model parameters are matched to physiological data: First, synaptic potentiation must be significantly stronger than synaptic depression for small (positive or negative) time lags between presynaptic and postsynaptic spikes. Second, spike synchronization must be sufficiently imprecise, for example, within a time window of 5–10 ms instead of 1 ms. Third, axonal propagation delays should not be much larger than dendritic delays. Under these assumptions synchronized neurons will be strongly coupled leading to a dominance of bidirectional synaptic connections even for simple STDP models and low mean firing rates at the level of spontaneous activity.

Electronic Implementation of a Fuzzy Neuron Model With a Gupta Integrator

In this paper the electronic circuit implementation of a fuzzy neuron model with a fuzzy Gupta integrator is presented.This neuron model simulates the performance and the fuzzy response of a fast-spiking biological neuron. The fuzzyneuron response is analyzed for two classical (non-fuzzy) input signals, the results are spike trains with relative andabsolute refractory period and an axonal delay. A comparison between the response of the proposed fuzzy neuronmodel and the intracellular registers of biological fast-spiking cortical interneurons is made, as well as the transientspresented at the beginning of each spike train. Also the results obtained from the electronic circuit of the fuzzy neuronmodel with the Matlab™ simulation of the mathematical model are compared.

Computation of Interaural Time Difference in the Owl's Coincidence Detector Neurons

Both the mammalian and avian auditory systems localize sound sources by computing the interaural time difference (ITD) with submillisecond accuracy. The neural circuits for this computation in birds consist of axonal delay lines and coincidence detector neurons. Here, we report the first in vivo intracellular recordings from coincidence detectors in the nucleus laminaris of barn owls. Binaural tonal stimuli induced sustained depolarizations (DC) and oscillating potentials whose waveforms reflected the stimulus. The amplitude of this sound analog potential (SAP) varied with ITD, whereas DC potentials did not. The amplitude of the SAP was correlated with firing rate in a linear fashion. Spike shape, synaptic noise, the amplitude of SAP, and responsiveness to current pulses differed between cells at different frequencies, suggesting an optimization strategy for sensing sound signals in neurons tuned to different frequencies.

Input-rate modulation of gamma oscillations is sensitive to network topology, delays and short-term plasticity

Simulated networks of excitatory and inhibitory neurons have previously been shown to reproduce critical features of experimental data regarding neural coding in V1, such as a positive relationship between thalamic input spike rate and the power of gamma frequency oscillations. This effect, referred to as modulated gamma power, could represent a neural code in V1 for stimulus characteristics that affect thalamic spike rate such as contrast or intensity. The simulated network's assumptions included homogeneous random connectivity, equal synaptic delays after spike arrival, and constant synaptic efficacies. Plausible alternative assumptions include small world connectivity, a wide distribution of axonal propagation delays, and short term synaptic plasticity, and here we assess the individual impact of each of these on the model's success in reproducing modulated gamma power. First, we developed several alternative algorithms for simulating directed networks with clustered connectivity and balanced excitation and inhibition. We found that modulated gamma power was absent in all small-world networks that had a relatively low abundance of reciprocal connectivity, which suggests that such motifs are present in V1 cortical networks at levels at least equal to those found in random networks. We also found in a different network type that the balance of excitation and inhibition could be destroyed when the network was in the small-world regime. Given all neurons had identical in-degrees, this result suggests that balance relies on motif distributions as well as mean connectivity. Second, altering the distribution of axonal delays had little effect, but increasing the mean delay led to a secondary gamma modulation at harmonics of the main peak, and since this is not observed experimentally, it suggests a mean delay in V1 networks less than 2ms. Finally, we compared two types of excitatory synaptic plasticity, and found that modulated beta power emerged in addition to gamma power for one type, in the presence of short term depression in interneurons.This article is part of a Special Issue entitled “Neural Coding”.

Neural Development of Binaural Tuning through Hebbian Learning Predicts Frequency-Dependent Best Delays

Birds use microsecond differences in the arrival times of the sounds at the two ears to infer the location of a sound source in the horizontal plane. These interaural time differences (ITDs) are encoded by binaural neurons which fire more when the ITD matches their "best delay." In the textbook model of sound localization, the best delays of binaural neurons reflect the differences in axonal delays of their monaural inputs, but recent observations have cast doubts on this classical view because best delays were found to depend on preferred frequency. Here, we show that these observations are in fact consistent with the notion that best delays are created by differences in axonal delays, provided ITD tuning is created during development through spike-timing-dependent plasticity: basilar membrane filtering results in correlations between inputs to binaural neurons, which impact the selection of synapses during development, leading to the observed distribution of best delays.

Cortical modulation of neuronal activity in the cat's lateral geniculate and perigeniculate nuclei: a modeling study

The role of cortical feedback in thalamo-cortical processing loop have been extensively investigated over the last decades. In general, these studies focused on cortical feedback exerted over lateral geniculate nucleus (LGN) principal cells, only in several cases the effects of cortical inactivation were investigated simultaneously in both thalamic relay cells and perigeniculate nucleus (PGN) inhibitory neurons. In the previous study [1] we showed in the cat that cessation of cortical input by cooling of visual cortex (areas 17 and 18) decreased spontaneous activity of LGN relay cells and increased spontaneous activity of PGN neurons. In contrast, visually evoked responses of most PGN neurons and LGN principal cells both decreased. To identify network mechanisms underlying such functional changes we conducted a modelling study in NEURON on several networks of point neurons with varied model parameters, such as membrane properties, synaptic weights and axonal delays. We considered five network topologies of the retino-geniculo-cortical pathway [2-6]. All models were robust against changes of axonal delays except for delay between LGN feed-forward interneuron and principal cell when this connection was present. In all such cases the models were found to be very sensitive to this delay. To reproduce the experimental results [1] the models required: 1) reciprocally connected PGN cells, connection present only in two model variants [2,3], and 2) slow decay of intracellular calcium. We found model [2] most consistent with known physiology and anatomy of the cat. Table ​Table1.1. Figure ​Figure11. Table 1 Mean spontaneous activity (spikes/s) for tested models (mean of 30 repetitions) and t-test P values. Figure 1 The model network of the thalamo-cortical loop following [2]. The model in [3] does not include LGN feedforward interneuron. Both variants reproduced experimental results [1]. White circles – relay cells, black – inhibitory neurons.

Maximized directed information transfer in critical neuronal networks

Critical dynamics in complex systems emerge at the transition from random to ordered dynamics and are characterized by power-law distributions of spatial and temporal properties of system events. The occurrence of critical dynamics in neuronal networks is increasingly supported by multielectrode array recordings of spontaneous activity in organotypic cortical slice cultures [1]. System events in these neuronal networks are typically defined as activations of neuronal ensembles, or “neuronal avalanches”. Interestingly, studies associate critical neuronal network avalanche dynamics with optimized information transfer [1,2]. However, studies have not previously examined the directed nature of information transfer in these networks. Here, we present three novel transfer-entropy [3] based measures of directed information transfer between neuronal avalanches. Our measures compute the amount of predictive information present in avalanches properties (avalanche size, avalanche duration and inter-avalanche period) of the source region about avalanche properties of the destination region and are suitable for detecting information transfer at multiple spatial scales, from individual neurons to neuronal ensembles. We apply these measures to compute directed information transfer in large, sparse, modular networks of leaky integrate-and-fire neurons with spike timing-dependent synaptic plasticity and axonal conduction delays. We characterize dynamics in our networks by distributions of neuronal avalanches and assess these distributions for power-law scaling. We compute directed information transfer between two halves of each network, and we normalize this transfer by the null-model transfer, generated by randomly rotating avalanche times for each avalanche vector, and so destroying any present relationships between groups of vectors. Dynamics in our networks change from subcritical to critical to supercritical as the modular topology of the networks is progressively randomized (Fig. ​(Fig.1a).1a). All three measures peak at criticality in all examined networks (Fig 1b-d) and hence show that directed neuronal information transfer is maximized at criticality in our model. These findings pave the way for the application of our measures to empirical multielectrode recording data. Figure 1 Directed information transfer at criticality. Fig 1A shows the presence of critical dynamics as a function of network randomization; p-values that exceed 0.5 imply power-law distributions of avalanches. Fig 1B-D show our normalized directed information ...

Effects of adaptation and synaptic plasticity on synchronization of coupled oscillating neurons

Synchronized oscillating activity in cortical circuits is a prevalent phenomenon and considered to be fundamental to cognitive function, selective attention and consciousness. In the present study we describe how spike-frequency adaptation (SFA) affects the synchronization properties of coupled neurons driven to repetitive firing, in dependence of axonal conduction delays and spike-timing-dependent plasticity (STDP). We analyze networks of adaptive exponential integrate-and-fire (aEIF) neurons by applying phase reduction theory based on the assumption of weak coupling [1]. We calculate infinitesimal phase response curves (PRCs) and combine these with conductance based AMPA- and GABA-like synapses, in order to obtain interaction functions which determine the dynamics of the reduced phase network. This allows for the identification of stable synchronous and out-of-phase locked states of coupled pairs. The aEIF model involves a subthreshold as well as a spike-triggered adaptation parameter which can be related to the M-current and the after hyperpolarization current, respectively, both producing SFA. The former causes a transition of the PRC from type I (only advancing) to type II (advancing and delaying) whereas the latter determines its skewness. We find that SFA in coupled excitatory pairs synchronizes oscillations as long as conduction delays are negligible, in agreement with [2]. When the delays are increased, the stable states shift from in-phase towards anti-phase locking (see Figure ​Figure1,1, middle). Inhibitory pairs show stable synchrony independent of the conduction delay, as well as anti-phase locking for short delays. Our analysis reveals that excitatory-inhibitory pairs do not phase lock if both neurons have type I PRCs. Type II pairs on the other hand have stable phase locked states even for diverging synaptic strengths (see Figure ​Figure1,1, bottom). When the coupling strengths are modified under the influence of a symmetric STDP rule, such pairs phase lock at a phase difference slightly above the conduction delay, for small delays. We complement these results on pairs by numerical simulations of phase as well as spiking aEIF networks, respectively, demonstrating that STDP promotes pairwise phase locking (clustering) in networks of excitatory neurons with low spike-triggered adaptation, where the number of clusters depends on the conduction delay. Figure 1 Top: Infinitesimal PRCs with low SFA (A), large subthreshold adaptation (B), large spike-triggered adaptation component (C). Middle: Stable (dark blue) and unstable (light blue) phase locked states of excitatory pairs in dependence of the conduction delay. ...

Neurobiologically Realistic Determinants of Self-Organized Criticality in Networks of Spiking Neurons

Self-organized criticality refers to the spontaneous emergence of self-similar dynamics in complex systems poised between order and randomness. The presence of self-organized critical dynamics in the brain is theoretically appealing and is supported by recent neurophysiological studies. Despite this, the neurobiological determinants of these dynamics have not been previously sought. Here, we systematically examined the influence of such determinants in hierarchically modular networks of leaky integrate-and-fire neurons with spike-timing-dependent synaptic plasticity and axonal conduction delays. We characterized emergent dynamics in our networks by distributions of active neuronal ensemble modules (neuronal avalanches) and rigorously assessed these distributions for power-law scaling. We found that spike-timing-dependent synaptic plasticity enabled a rapid phase transition from random subcritical dynamics to ordered supercritical dynamics. Importantly, modular connectivity and low wiring cost broadened this transition, and enabled a regime indicative of self-organized criticality. The regime only occurred when modular connectivity, low wiring cost and synaptic plasticity were simultaneously present, and the regime was most evident when between-module connection density scaled as a power-law. The regime was robust to variations in other neurobiologically relevant parameters and favored systems with low external drive and strong internal interactions. Increases in system size and connectivity facilitated internal interactions, permitting reductions in external drive and facilitating convergence of postsynaptic-response magnitude and synaptic-plasticity learning rate parameter values towards neurobiologically realistic levels. We hence infer a novel association between self-organized critical neuronal dynamics and several neurobiologically realistic features of structural connectivity. The central role of these features in our model may reflect their importance for neuronal information processing.

Effect of the Topology and Delayed Interactions in Neuronal Networks Synchronization

As important as the intrinsic properties of an individual nervous cell stands the network of neurons in which it is embedded and by virtue of which it acquires great part of its responsiveness and functionality. In this study we have explored how the topological properties and conduction delays of several classes of neural networks affect the capacity of their constituent cells to establish well-defined temporal relations among firing of their action potentials. This ability of a population of neurons to produce and maintain a millisecond-precise coordinated firing (either evoked by external stimuli or internally generated) is central to neural codes exploiting precise spike timing for the representation and communication of information. Our results, based on extensive simulations of conductance-based type of neurons in an oscillatory regime, indicate that only certain topologies of networks allow for a coordinated firing at a local and long-range scale simultaneously. Besides network architecture, axonal conduction delays are also observed to be another important factor in the generation of coherent spiking. We report that such communication latencies not only set the phase difference between the oscillatory activity of remote neural populations but determine whether the interconnected cells can set in any coherent firing at all. In this context, we have also investigated how the balance between the network synchronizing effects and the dispersive drift caused by inhomogeneities in natural firing frequencies across neurons is resolved. Finally, we show that the observed roles of conduction delays and frequency dispersion are not particular to canonical networks but experimentally measured anatomical networks such as the macaque cortical network can display the same type of behavior.