Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks

Yajing Zheng,Shanshan Jia,Zhaofei Yu,Jian K Liu,Tiejun Huang

doi:10.1016/j.patter.2021.100350

Abstract

SummaryTraditional models of retinal system identification analyze the neural response to artificial stimuli using models consisting of predefined components. The model design is limited to prior knowledge, and the artificial stimuli are too simple to be compared with stimuli processed by the retina. To fill in this gap with an explainable model that reveals how a population of neurons work together to encode the larger field of natural scenes, here we used a deep-learning model for identifying the computational elements of the retinal circuit that contribute to learning the dynamics of natural scenes. Experimental results verify that the recurrent connection plays a key role in encoding complex dynamic visual scenes while learning biological computational underpinnings of the retinal circuit. In addition, the proposed models reveal both the shapes and the locations of the spatiotemporal receptive fields of ganglion cells.

Highlights

Unraveling the neural system of the brain is one of the key questions of both neuroscience and artificial intelligence, as understanding the structure of neural systems could help to develop novel methodologies of artificial intelligence
Some recent studies have explored the role of recurrence,[39] using recurrent neural networks (RNNs) to model the shared feature space within the population of neurons. The performance of this approach depends critically on the initial location estimate. To fill in this gap with an explainable model that reveals how the population of neurons work together to encode a larger field of dynamic natural scenes, in this study, we propose that the computations carried out by the retina could be better explained by a convolutional RNN (CRNN) rather than a feedforward convolutional neural networks (CNNs)
To better study the working principle of the encoding of external input stimuli by the retina, we introduce recurrent connections based on a CNN to get closer to the anatomy of the retina, i.e., the lateral connection, between the retinal ganglion cells (RGCs) by gap junction or amacrine cells

Summary

Introduction

Unraveling the neural system of the brain is one of the key questions of both neuroscience and artificial intelligence, as understanding the structure of neural systems could help to develop novel methodologies of artificial intelligence. The visual system constantly receives highly complex and dynamic visual scenes with a high order of spatiotemporal correlations To cope with these inputs, it is necessary to develop an explainable neural network model, either for explaining the data of neuroscience, e.g., the neural response to input scenes,[1] or for developing an efficient computational framework for analyzing dynamic visual scenes for artificial vision.[2]. At the output side of the retina, i.e., the retinal ganglion cells (RGCs), all input signals are transformed into a sequence of spikes. These spikes are transmitted via the optic nerve to the visual processing center of the brain. Exploring the encoding mechanism of the retina is essential to unravel the computational principles of other visual systems

Results

Discussion

Conclusion