Building a brain with SYCL and modern C++

Abstract

State-of-the art machine learning systems typically depend on energetically costly gradient-descent learning over a curated task-specific data set. Despite their successes, these methods are not well suited to building fully autonomous systems such as may employ energy-efficient accelerators targeted by OpenCL. By contrast, the brain uses low-energy local learning rules to discover the causal structure of an environment, forming semantically rich representations without supervision, and therefore exhibiting the required combination of efficiency and flexibility. To investigate these properties, a paradigm shift to dynamic spike-based computation is required. Historically, investigating spiking neural models has been a task for specialists, with software that is tailored to specific scientific projects, or that trades flexibility against performance. Here, we present neurosycl, a high-performance, portable spiking network simulator based on SYCL, with a modern and extensible C++ API. Our aim is to provide the necessary components for non-specialists to build a simulated brain, and to run the constructed models as close to real-time as possible.This bipartite aim leads to two competing considerations -- a simple interface, and portable performance -- which are reconciled using SYCL's single-source programming model. We describe two principal algorithmic challenges that illustrate the different hardware demands of spiking neural networks relative to deep learning networks, and how neurosycl solves them for GPU-like parallel processors via SYCL. Firstly, although the brain is akin to a parallel processor whose cores are neurons, the connections between neurons may have differing temporal delays, which results in a message-passing problem if the neurons are simulated asynchronously. Secondly, because these messages ('spikes') are generated chaotically, then transmitted to arbitrary target neurons with arbitrary transmission delays, a naive implementation even of a synchronous model quickly runs into a highly suboptimal memory access regime.neurosycl's design separates the specification of a model architecture from its simulation, so that once a model has been instantiated, its basic structure is fixed. This simplification enables us to infer the memory access pattern, and thus re-order the connection indices so that adjacent kernels access nearby memory locations. The simplification is also reflected in the API design: users can construct complex connection graphs between arbitrary neuron groups using a simple declarative interface, but runtime interactions with the model, for monitoring or I/O, are mediated by a set of simulated electrodes, combined with hooks into the simulation loop. This design mirrors that of neuroscientific experiments, and permits the user to embed the simulated brain into a virtual environment by integrating with other technologies, exposing implementation details only when necessary to allow this. We describe our API, illustrated by a number of brain-building examples, showing how the components compose and map via SYCL onto the hardware. We present performance comparisons across hardware platforms and alternative simulators, demonstrating portability for various network configurations and standard neuron models.