Coarse-Grained Neural Network Model of the Basal Ganglia to Simulate Reinforcement Learning Tasks.

Dorota Frydecka,Jarosław Drapała

doi:10.3390/brainsci12020262

Abstract

Computational models of the basal ganglia (BG) provide a mechanistic account of different phenomena observed during reinforcement learning tasks performed by healthy individuals, as well as by patients with various nervous or mental disorders. The aim of the present work was to develop a BG model that could represent a good compromise between simplicity and completeness. Based on more complex (fine-grained neural network, FGNN) models, we developed a new (coarse-grained neural network, CGNN) model by replacing layers of neurons with single nodes that represent the collective behavior of a given layer while preserving the fundamental anatomical structures of BG. We then compared the functionality of both the FGNN and CGNN models with respect to several reinforcement learning tasks that are based on BG circuitry, such as the Probabilistic Selection Task, Probabilistic Reversal Learning Task and Instructed Probabilistic Selection Task. We showed that CGNN still has a functionality that mirrors the behavior of the most often used reinforcement learning tasks in human studies. The simplification of the CGNN model reduces its flexibility but improves the readability of the signal flow in comparison to more detailed FGNN models and, thus, can help to a greater extent in the translation between clinical neuroscience and computational modeling.

Highlights

The basal ganglia (BG) are a set of subcortical nuclei responsible primarily for motor control [1]; they play roles in motor learning, executive functions, emotional processing and action inhibition
The BG have been wildly researched within the framework of cognitive neuroscience in an attempt to gain a deeper understanding of the neuronal basis of psychiatric and neurological disorders [2]
Accumulating evidence suggests that the contribution of various BG components may be described within a reinforcement learning model [5]

Summary

Introduction

The basal ganglia (BG) are a set of subcortical nuclei responsible primarily for motor control [1]; they play roles in motor learning, executive functions, emotional processing and action inhibition. Many computational models have been proposed by researchers with the aim of studying the internal structure and functioning of the BG and providing deeper insight into the outcomes of experiments involving human subjects [3,4]. The reinforcement signal, referred to as a reward prediction error signal, is transferred from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) to the BG through the dopaminergic neurons that fire in proportion to the difference between the expected and actual reward [8]. The error signal from midbrain dopaminergic neurons, together with environmental cues from the cortex, create convergent information that modifies the activity of the striatum [9]. The dynamic modulation of striatal activity is causally related to behavioral changes [10]

Objectives

Methods

Results

Conclusion