Sparse Long-range Connectivity Research Articles

Self-Organized Structuring of Recurrent Neuronal Networks for Reliable Information Transmission.

Simple SummaryInformation processing in the brain takes places at multiple stages, each of which is a local network of neurons. The long-range connections between these network stages are sparse and do not change over time. Thus, within each stage information arrives at a sparse subset of input neurons and must be routed to a sparse subset of output neurons. In this theoretical work, we investigate how networks achieve this routing in a self-organized manner without losing information. We show that biologically inspired self-organization entails that input information is distributed to all neurons in the network by strengthening many synapses in the local networks. Thus, after successful self-organization, input information can be read out and decoded from a small number of outputs. We also show that this way of self-organization can still be more energy efficient than creating more long-range in- and output connections.Our brains process information using a layered hierarchical network architecture, with abundant connections within each layer and sparse long-range connections between layers. As these long-range connections are mostly unchanged after development, each layer has to locally self-organize in response to new inputs to enable information routing between the sparse in- and output connections. Here we demonstrate that this can be achieved by a well-established model of cortical self-organization based on a well-orchestrated interplay between several plasticity processes. After this self-organization, stimuli conveyed by sparse inputs can be rapidly read out from a layer using only very few long-range connections. To achieve this information routing, the neurons that are stimulated form feed-forward projections into the unstimulated parts of the same layer and get more neurons to represent the stimulus. Hereby, the plasticity processes ensure that each neuron only receives projections from and responds to only one stimulus such that the network is partitioned into parts with different preferred stimuli. Along this line, we show that the relation between the network activity and connectivity self-organizes into a biologically plausible regime. Finally, we argue how the emerging connectivity may minimize the metabolic cost for maintaining a network structure that rapidly transmits stimulus information despite sparse input and output connectivity.

Neural Classifiers with Limited Connectivity and Recurrent Readouts.

For many neural network models in which neurons are trained to classify inputs like perceptrons, the number of inputs that can be classified is limited by the connectivity of each neuron, even when the total number of neurons is very large. This poses the problem of how the biological brain can take advantage of its huge number of neurons given that the connectivity is sparse. One solution is to combine multiple perceptrons together, as in committee machines. The number of classifiable random patterns would then grow linearly with the number of perceptrons, even when each perceptron has limited connectivity. However, the problem is moved to the downstream readout neurons, which would need a number of connections as large as the number of perceptrons. Here we propose a different approach in which the readout is implemented by connecting multiple perceptrons in a recurrent attractor neural network. We prove analytically that the number of classifiable random patterns can grow unboundedly with the number of perceptrons, even when the connectivity of each perceptron remains finite. Most importantly, both the recurrent connectivity and the connectivity of downstream readouts also remain finite. Our study shows that feedforward neural classifiers with numerous long-range afferent connections can be replaced by recurrent networks with sparse long-range connectivity without sacrificing the classification performance. Our strategy could be used to design more general scalable network architectures with limited connectivity, which resemble more closely the brain neural circuits that are dominated by recurrent connectivity.SIGNIFICANCE STATEMENT The mammalian brain has a huge number of neurons, but the connectivity is rather sparse. This observation seems to contrast with the theoretical studies showing that for many neural network models the performance scales with the number of connections per neuron and not with the total number of neurons. To solve this dilemma, we propose a model in which a recurrent network reads out multiple neural classifiers. Its performance scales with the total number of neurons even when each neuron of the network has limited connectivity. Our study reveals an important role of recurrent connections in neural systems like the hippocampus, in which the computational limitations due to sparse long-range feedforward connectivity might be compensated by local recurrent connections.

Functional Interactions between Mammalian Respiratory Rhythmogenic and Premotor Circuitry.

Breathing in mammals depends on rhythms that originate from the preBötzinger complex (preBötC) of the ventral medulla and a network of brainstem and spinal premotor neurons. The rhythm-generating core of the preBötC, as well as some premotor circuits, consist of interneurons derived from Dbx1-expressing precursors (Dbx1 neurons), but the structure and function of these networks remain incompletely understood. We previously developed a cell-specific detection and laser ablation system to interrogate respiratory network structure and function in a slice model of breathing that retains the preBötC, the respiratory-related hypoglossal (XII) motor nucleus and XII premotor circuits. In spontaneously rhythmic slices, cumulative ablation of Dbx1 preBötC neurons decreased XII motor output by ∼50% after ∼15 cell deletions, and then decelerated and terminated rhythmic function altogether as the tally increased to ∼85 neurons. In contrast, cumulatively deleting Dbx1 XII premotor neurons decreased motor output monotonically but did not affect frequency nor stop XII output regardless of the ablation tally. Here, we couple an existing preBötC model with a premotor population in several topological configurations to investigate which one may replicate the laser ablation experiments best. If the XII premotor population is a "small-world" network (rich in local connections with sparse long-range connections among constituent premotor neurons) and connected with the preBötC such that the total number of incoming synapses remains fixed, then the in silico system successfully replicates the in vitro laser ablation experiments. This study proposes a feasible configuration for circuits consisting of Dbx1-derived interneurons that generate inspiratory rhythm and motor pattern. To produce a breathing-related motor pattern, a brainstem core oscillator circuit projects to a population of premotor interneurons, but the assemblage of this network remains incompletely understood. Here we applied network modeling and numerical simulation to discover respiratory circuit configurations that successfully replicate photonic cell ablation experiments targeting either the core oscillator or premotor network, respectively. If premotor neurons are interconnected in a so-called "small-world" network with a fixed number of incoming synapses balanced between premotor and rhythmogenic neurons, then our simulations match their experimental benchmarks. These results provide a framework of experimentally testable predictions regarding the rudimentary structure and function of respiratory rhythm- and pattern-generating circuits in the brainstem of mammals.

Weak signal detection and propagation in diluted feed-forward neural network with recurrent excitation and inhibition

Reliable signal propagation across distributed brain areas provides the basis for neural circuit function. Modeling studies on cortical circuits have shown that multilayered feed-forward networks (FFNs), if strongly and/or densely connected, can enable robust signal propagation. However, cortical networks are typically neither densely connected nor have strong synapses. This paper investigates under which conditions spiking activity can be propagated reliably across diluted FFNs. Extending previous works, we model each layer as a recurrent sub-network constituting both excitatory (E) and inhibitory (I) neurons and consider the effect of interactions between local excitation and inhibition on signal propagation. It is shown that elevation of cellular excitation–inhibition (EI) balance in the local sub-networks (layers) softens the requirement for dense/strong anatomical connections and thereby promotes weak signal propagation in weakly connected networks. By means of iterated maps, we show how elevated local excitability state compensates for the decreased gain of synchrony transfer function that is due to sparse long-range connectivity. Finally, we report that modulations of EI balance and background activity provide a mechanism for selectively gating and routing neural signal. Our results highlight the essential role of intrinsic network states in neural computation.

A wavelet-based neural model to optimize and read out a temporal population code

It has been proposed that the dense excitatory local connectivity of the neo-cortex plays a specific role in the transformation of spatial stimulus information into a temporal representation or a temporal population code (TPC). TPC provides for a rapid, robust, and high-capacity encoding of salient stimulus features with respect to position, rotation, and distortion. The TPC hypothesis gives a functional interpretation to a core feature of the cortical anatomy: its dense local and sparse long-range connectivity. Thus far, the question of how the TPC encoding can be decoded in downstream areas has not been addressed. Here, we present a neural circuit that decodes the spectral properties of the TPC using a biologically plausible implementation of a Haar transform. We perform a systematic investigation of our model in a recognition task using a standardized stimulus set. We consider alternative implementations using either regular spiking or bursting neurons and a range of spectral bands. Our results show that our wavelet readout circuit provides for the robust decoding of the TPC and further compresses the code without loosing speed or quality of decoding. We show that in the TPC signal the relevant stimulus information is present in the frequencies around 100 Hz. Our results show that the TPC is constructed around a small number of coding components that can be well decoded by wavelet coefficients in a neuronal implementation. The solution to the TPC decoding problem proposed here suggests that cortical processing streams might well consist of sequential operations where spatio-temporal transformations at lower levels forming a compact stimulus encoding using TPC that are subsequently decoded back to a spatial representation using wavelet transforms. In addition, the results presented here show that different properties of the stimulus might be transmitted to further processing stages using different frequency components that are captured by appropriately tuned wavelet-based decoders.

Convergence Speed of the Consensus Algorithm With Interference and Sparse Long-Range Connectivity

We analyze the effect of interference on the convergence rate of average consensus algorithms, which iteratively compute the measurement average by message passing among nodes. It is usually assumed that these algorithms converge faster with a greater exchange of information (i.e., by increased network connectivity) in every iteration. However, when interference is taken into account, it is no longer clear if the rate of convergence increases with network connectivity. We study this problem for randomly-placed consensus-seeking nodes connected through an interference-limited network. We investigate the following questions: (a) How does the rate of convergence vary with increasing communication range of each node? and (b) How does this result change when each node is allowed to communicate with a few selected far-off nodes? When nodes schedule their transmissions to avoid interference, we show that the convergence speed scales with $r^{2-d}$, where $r$ is the communication range and $d$ is the number of dimensions. This scaling is the result of two competing effects when increasing $r$: Increased schedule length for interference-free transmission vs. the speed gain due to improved connectivity. Hence, although one-dimensional networks can converge faster from a greater communication range despite increased interference, the two effects exactly offset one another in two-dimensions. In higher dimensions, increasing the communication range can actually degrade the rate of convergence. Our results thus underline the importance of factoring in the effect of interference in the design of distributed estimation algorithms.

Geometric phase-transition on systems with sparse long-range connections

Small-world networks are regular structures with a fraction p of regular connections per site replaced by totally random ones (“shortcuts”). This kind of structure seems to be present on networks arising in nature and technology. In this work we show that the small-world transition is a first-order transition at zero density p of shortcuts, whereby the normalized shortest-path distance L= ℓ ̄ /L undergoes a discontinuity in the thermodynamic limit. On finite systems the apparent transition is shifted by Δ p∼ L − d . Equivalently a “persistence size” L ∗∼p −1/d can be defined in connection with finite-size effects. Assuming L ∗∼p −τ , simple rescaling arguments imply that τ=1/ d. We confirm this result by extensive numerical simulation in one to four dimensions, and argue that τ=1/ d implies that this transition is first-order.

Spreading and shortest paths in systems with sparse long-range connections.

Spreading according to simple rules (e.g., of fire or diseases) and shortest-path distances are studied on d-dimensional systems with a small density p per site of long-range connections ("small-world" lattices). The volume V(t) covered by the spreading quantity on an infinite system is exactly calculated in all dimensions as a function of time t. From this, the average shortest-path distance l(r) can be calculated as a function of Euclidean distance r. It is found that l(r) approximately r for r<r(c)=[2p Gamma(d)(d-1)!](-1/d) log(2p Gamma(d)L(d)) and l(r) approximately r(c) for r>r(c). The characteristic length r(c), which governs the behavior of shortest-path lengths, diverges logarithmically with L for all p>0.