A Non-spiking Neuron Model With Dynamic Leak to Avoid Instability in Recurrent Networks.

Udaya B Rongala,Jonas M D Enander,Henrik Jörntell,Matthias Kohler,Gerald E Loeb

doi:10.3389/fncom.2021.656401

Abstract

Recurrent circuitry components are distributed widely within the brain, including both excitatory and inhibitory synaptic connections. Recurrent neuronal networks have potential stability problems, perhaps a predisposition to epilepsy. More generally, instability risks making internal representations of information unreliable. To assess the inherent stability properties of such recurrent networks, we tested a linear summation, non-spiking neuron model with and without a “dynamic leak”, corresponding to the low-pass filtering of synaptic input current by the RC circuit of the biological membrane. We first show that the output of this neuron model, in either of its two forms, follows its input at a higher fidelity than a wide range of spiking neuron models across a range of input frequencies. Then we constructed fully connected recurrent networks with equal numbers of excitatory and inhibitory neurons and randomly distributed weights across all synapses. When the networks were driven by pseudorandom sensory inputs with varying frequency, the recurrent network activity tended to induce high frequency self-amplifying components, sometimes evident as distinct transients, which were not present in the input data. The addition of a dynamic leak based on known membrane properties consistently removed such spurious high frequency noise across all networks. Furthermore, we found that the neuron model with dynamic leak imparts a network stability that seamlessly scales with the size of the network, conduction delays, the input density of the sensory signal and a wide range of synaptic weight distributions. Our findings suggest that neuronal dynamic leak serves the beneficial function of protecting recurrent neuronal circuitry from the self-induction of spurious high frequency signals, thereby permitting the brain to utilize this architectural circuitry component regardless of network size or recurrency.

Highlights

Recurrent excitatory loops are a common feature in the central nervous system, such as in neocortical circuits (Binzegger et al, 2004; Song et al, 2005; Koestinger et al, 2018; Kar and DiCarlo, 2020), thalamocortical loops (Steriade, 1997; Hooks et al, 2013), cerebrocerebellar and spinocerebellar loops (Allen and Tsukahara, 1974; Jörntell, 2017)
The spikes generated by the Izhikevich Neuron Model (IZ) neuron model were convoluted to a time continuous signal (Figure 3E) in order for it to be comparable with the output of the Linear Summation Neuron Model (LSM)
The LSM neuron without dynamic leak reproduced on average a close representation of the source convolution signal for the input but the individual traces were considerably noisier without dynamic leak (Figure 3C) than with dynamic leak (Figure 3D)

Summary

Introduction

Recurrent excitatory loops are a common feature in the central nervous system, such as in neocortical circuits (Binzegger et al, 2004; Song et al, 2005; Koestinger et al, 2018; Kar and DiCarlo, 2020), thalamocortical loops (Steriade, 1997; Hooks et al, 2013), cerebrocerebellar and spinocerebellar loops (Allen and Tsukahara, 1974; Jörntell, 2017). Neuron Leak Stabilizes Recurrent Networks been described to provide lateral inhibition (Zhu and Lo, 2000; Douglas and Martin, 2009; Obermayer et al, 2018; Rongala et al, 2018) and feed-forward inhibition (Swadlow, 2003; Isaacson and Scanziani, 2011), but they make synapses on other inhibitory neurons, thereby potentially forming recurrent disinhibitory loops as well (Jörntell and Ekerot, 2003; Pi et al, 2013; Sultan and Shi, 2018). Due to the many potential positive feedback loops in larger networks with extensive recurrent connections, imbalances in excitatory (E) and inhibitory (I) synaptic activity could lead to activity saturation (Brunel, 2000; Vogels and Abbott, 2005), such as observed in epilepsy (Chakravarthy et al, 2009; Liou et al, 2020), or, in milder cases, a noise-like perturbation of the information content of internal signals, which would be disadvantageous for learning

Methods

Results

Conclusion