Representation learning for neural population activity with Neural Data Transformers

Joel Ye,Chethan Pandarinath

doi:10.51628/001c.27358

Joel Ye, Chethan Pandarinath

Open Access

https://doi.org/10.51628/001c.27358

Copy DOI

Abstract

Neural population activity is theorized to reflect an underlying dynamical structure. This structure can be accurately captured using state space models with explicit dynamics, such as those based on recurrent neural networks (RNNs). However, using recurrence to explicitly model dynamics necessitates sequential processing of data, slowing real-time applications such as brain-computer interfaces. Here we introduce the Neural Data Transformer (NDT), a non-recurrent alternative. We test the NDT’s ability to capture autonomous dynamical systems by applying it to synthetic datasets with known dynamics and data from monkey motor cortex during a reaching task well-modeled by RNNs. The NDT models these datasets as well as state-of-the-art recurrent models. Further, its non-recurrence enables 3.9ms inference, well within the loop time of real-time applications and more than 6 times faster than recurrent baselines on the monkey reaching dataset. These results suggest that an explicit dynamics model is not necessary to model autonomous neural population dynamics. Code: https://github.com/snel-repo/neural-data-transformers

Highlights

Neural populations are theorized to have an underlying dynamical structure which drives the evolution of population activity over time [1, 2, 3]
latent factor analysis via dynamical systems (LFADS) is known to benefit from Population-Based Training (PBT [21]) over simple grid search. (We find that Neural Data Transformer (NDT) performs comparably between grid search and PBT.) AutoLFADS PBT is run with exponentially-smoothed validation NLL as the exploitation metric, and so we select the least smoothed validation NLL checkpoint [12]
We apply AutoLFADS with fixed settings that were previously shown to work in a variety of applications [12]

Summary

Introduction

Neural populations are theorized to have an underlying dynamical structure which drives the evolution of population activity over time [1, 2, 3]. An RNN-based method called latent factor analysis via dynamical systems (LFADS) has been shown to model single trial variability in neural spiking activity far better than traditional baselines like spike smoothing or GPFA [9, 12]. This precise modeling enables accurate prediction of subjects’ behaviors on a moment-by-moment basis and millisecond timescale. Though neuroscience datasets may not yet be large enough to realize much training benefit, reduced inference times could already benefit real-time applications where cycle times are critical, such as brain-computer interfaces or closed-loop neural stimulation

Objectives

Results

Conclusion