Just-in-time connectivity for very large neuronal networks

Abstract

Memory storage remains a limitation when running very large neuronal networks (VLNN). With number of neurons n, connectivity storage grows as n2. With connectivity densities of 0.1–10%, 1e6 neurons will require 1e9-1e11 synapses. A single connection requires at least an associated weight and delay as well as additional pointers or offsets to store the connectivity matrix. Conservatively, this will require 10 bytes (e.g., 2 floats and 2 chars) which will then bring the total synaptic memory load to 10 GB-1 TB. The former value may be barely executable on a single large machine. We have exploited an algorithmic space-time trade-off to build large event-driven artificial-cell simulations in the NEURON simulator by utilizing just-in-time connections (JITCONs) that are generated at the time of presynaptic cell spiking. JITCON utilizes a presynaptic-cell-specific random-number-generator seed based on presynaptic-cell serial number that permits it to generate a list of postsynaptic cell targets on the fly, and seeds based on a multiple of presynaptic-cell and postsynaptic-cell serial numbers for generating weights. We have utilized the JITCON algorithm to readily run simulations of >2e6 neurons. These simulations include a moderate level of cellular detail with AMPA, NMDA, GABAA and GABAB synapses, as well as multiple intrinsic properties such as bursting, depolarization blockade and an afterhyperpolarizing channel. Note that these are event-driven simulations and therefore do not utilize continuously integrated compartmental neurons. Since these simulations are event-driven, there is no overhead unless there is activity: simulation time varies widely depending on the level of network activity. An active network of 1.2e5 cells with >8.9e6 synapses, generating >1.1e7 spikes in 1 s simulation time, took 32.3 minutes to run on a 2.4 GHz AMD Opteron processor. Large, active simulations still develop space problems due to the need for a variable-size queue to accommodate varying delivery delays. This limitation is minimized by restrictions on the range and variability of permitted delays. We have begun to explore algorithms that permit a nuanced approach to the space-time trade-off. We permit individual presynaptic cells to store their list of postsynaptic targets in a compressed format. This additional storage can be turned on or off on a per-cell basis. We will explore making this storage dynamic so that a cell can maintain its connectivity list during a period of high activity and then return the memory when its activity is reduced. An additional direction for future development will be the incorporation of an entire encapsulated artificial-cell network as an independent piece of a compiled code (a mod file in NEURON). Such a network module could then be plugged in to other network modules or to a more detailed network that used compartmental models or compartmental/artificial cell hybrids, running in the main NEURON simulator. Running such simulations on parallel supercomputers will permit execution of very-VLNNs of order 100 million neurons.