Computational Model Of Basal Ganglia Research Articles

Stereotyped, automatized and habitual behaviours: are they similar constructs under the control of the same cerebral areas?

Comprehensive knowledge about higher executive functions of motor control has been covered in the last decades. Critical goals have been targeted through many different technological approaches. An abundant flow of new results greatly progressed our ability to respond at better-posited answers to look more than ever at the challenging neural system functioning. Behaviour is the observable result of the invisible, as complex cerebral functioning. Many pathological states are approached after symptomatology categorisation of behavioural impairments is achieved. Motor, non-motor and psychiatric signs are greatly shared by many neurological/psychiatric disorders. Together with the cerebral cortex, the basal ganglia contribute to the expression of behaviour promoting the correct action schemas and the selection of appropriate sub-goals based on the evaluation of action outcomes. The present review focus on the basic classification of higher motor control functioning, taking into account the recent advances in basal ganglia structural knowledge and the computational model of basal ganglia functioning. We discuss about the basal ganglia capability in executing ordered motor patterns in which any single movement is linked to each other into an action, and many actions are ordered into each other, giving them a syntactic value to the final behaviour. The stereotypic, automatized and habitual behaviour's constructs and controls are the expression of successive stages of rule internalization and categorisation aimed in producing the perfect spatial-temporal control of motor command.

Bipolar oscillations between positive and negative mood states in a computational model of Basal Ganglia.

Bipolar disorder is characterized by mood swings-oscillations between manic and depressive states. The swings (oscillations) mark the length of an episode in a patient's mood cycle (period), and can vary from hours to years. The proposed modeling study uses decision making framework to investigate the role of basal ganglia network in generating bipolar oscillations. In this model, the basal ganglia system performs a two-arm bandit task in which one of the arms (action responses) leads to a positive outcome, while the other leads to a negative outcome. We explore the dynamics of key reward and risk related parameters in the system while the model agent receives various outcomes. Particularly, we study the system using a model that represents the fast dynamics of decision making, and a module to capture the slow dynamics that describe the variation of some meta-parameters of fast dynamics over long time scales. The model is cast at three levels of abstraction: (1) a two-dimensional dynamical system model, that is a simple two variable model capable of showing bistability for rewarding and punitive outcomes; (2) a phenomenological basal ganglia model, to extend the implications from the reduced model to a cortico-basal ganglia setup; (3) a detailed network model of basal ganglia, that incorporates detailed cellular level models for a more realistic understanding. In healthy conditions, the model chooses positive action and avoids negative one, whereas under bipolar conditions, the model exhibits slow oscillations in its choice of positive or negative outcomes, reminiscent of bipolar oscillations. Phase-plane analyses on the simple reduced dynamical system with two variables reveal the essential parameters that generate pathological 'bipolar-like' oscillations. Phenomenological and network models of the basal ganglia extend that logic, and interpret bipolar oscillations in terms of the activity of dopaminergic and serotonergic projections on the cortico-basal ganglia network dynamics. The network's dysfunction, specifically in terms of reward and risk sensitivity, is shown to be responsible for the pathological bipolar oscillations. The study proposes a computational model that explores the effects of impaired serotonergic neuromodulation on the dynamics of the cortico basal ganglia network, and relates this impairment to abstract mood states (manic and depressive episodes) and oscillations of bipolar disorder.

Probing the Role of Medication, DBS Electrode Position, and Antidromic Activation on Impulsivity Using a Computational Model of Basal Ganglia.

Everyday, we encounter situations where available choices are nearly equally rewarding (high conflict) calling for some tough decision making. Experimental recordings showed that the activity of Sub Thalamic Nucleus (STN) increases during such situations providing the extra time needed to make the right decision, teasing apart the most rewarding choice from the runner up closely trailing behind. This prolonged deliberation necessary for decision making under high conflict was absent in Parkinson's disease (PD) patients who underwent Deep Brain Stimulation (DBS) surgery of STN. In an attempt to understand the underlying cause of such adverse response, we built a 2D spiking network model (50 × 50 lattice) of Basal ganglia incorporating the key nuclei. Using the model we studied the Probabilistic learning task (PLT) in untreated, treated (L-Dopa and Dopamine Agonist) and STN-DBS PD conditions. Based on the experimental observation that dopaminergic activity is analogous to temporal difference (TD) and induces cortico-striatal plasticity, we introduced learning in the cortico-striatal weights. The results show that healthy and untreated conditions of PD model were able to more or less equally select (avoid) the rewarding (punitive) choice, a behavior that was absent in treated PD condition. The time taken to select a choice in high conflict trials was high in normal condition, which is in agreement with experimental results. The treated PD (Dopamine Agonist) patients made impulsive decisions (small reaction time) which in turn led to poor performance. The underlying cause of the observed impulsivity in DBS patients was studied in the model by (1) varying the electrode position within STN, (2) causing antidromic activation of GPe neurons. The effect of electrode position on reaction time was analyzed by studying the activity of STN neurons where, a decrease in STN neural activity was observed for certain electrode positions. We also observed that a higher antidromic activation of GPe neurons does not impact the learning ability but decreases reaction time as reported in DBS patients. These results suggest a probable role of electrode and antidromic activation in modulating the STN activity and eventually affecting the patient's performance on PLT.

A Biologically Inspired Computational Model of Basal Ganglia in Action Selection.

The basal ganglia (BG) are a subcortical structure implicated in action selection. The aim of this work is to present a new cognitive neuroscience model of the BG, which aspires to represent a parsimonious balance between simplicity and completeness. The model includes the 3 main pathways operating in the BG circuitry, that is, the direct (Go), indirect (NoGo), and hyperdirect pathways. The main original aspects, compared with previous models, are the use of a two-term Hebb rule to train synapses in the striatum, based exclusively on neuronal activity changes caused by dopamine peaks or dips, and the role of the cholinergic interneurons (affected by dopamine themselves) during learning. Some examples are displayed, concerning a few paradigmatic cases: action selection in basal conditions, action selection in the presence of a strong conflict (where the role of the hyperdirect pathway emerges), synapse changes induced by phasic dopamine, and learning new actions based on a previous history of rewards and punishments. Finally, some simulations show model working in conditions of altered dopamine levels, to illustrate pathological cases (dopamine depletion in parkinsonian subjects or dopamine hypermedication). Due to its parsimonious approach, the model may represent a straightforward tool to analyze BG functionality in behavioral experiments.

A computational model of altered gait patterns in parkinson's disease patients negotiating narrow doorways

We present a computational model of altered gait velocity patterns in Parkinson's Disease (PD) patients. PD gait is characterized by short shuffling steps, reduced walking speed, increased double support time and sometimes increased cadence. The most debilitating symptom of PD gait is the context dependent cessation in gait known as freezing of gait (FOG). Cowie et al. (2010) and Almeida and Lebold (2010) investigated FOG as the changes in velocity profiles of PD gait, as patients walked through a doorway with variable width. The former reported a sharp dip in velocity, a short distance from the doorway that was greater for narrower doorways. They compared the gait performance in PD freezers at ON and OFF dopaminergic medication. In keeping with this finding, the latter also reported the same for ON medicated PD freezers and non-freezers. In the current study, we sought to simulate these gait changes using a computational model of Basal Ganglia based on Reinforcement Learning, coupled with a spinal rhythm mimicking central pattern generator (CPG) model. In the model, a simulated agent was trained to learn a value profile over a corridor leading to the doorway by repeatedly attempting to pass through the doorway. Temporal difference error in value, associated with dopamine signal, was appropriately constrained in order to reflect the dopamine-deficient conditions of PD. Simulated gait under PD conditions exhibited a sharp dip in velocity close to the doorway, with PD OFF freezers showing the largest decrease in velocity compared to PD ON freezers and controls. PD ON and PD OFF freezers both showed sensitivity to the doorway width, with narrow door producing the least velocity/ stride length. Step length variations were also captured with PD freezers producing smaller steps and larger step-variability than PD non-freezers and controls. In addition this model is the first to explain the non-dopamine dependence for FOG giving rise to several other possibilities for its etiology.

A computational model of the basal ganglia dual pathways organization

We elaborated computational models of the whole primate basal ganglia (BG) with the aim of answering to the long standing questions [1,2] of the relationships between the external part of the globus pallidus (GPe) and its internal part (GPi). In particular, we aimed to solve the apparent contradiction between a) anatomical studies showing that the GPi is an important target of a gabaergic inhibition from GPe [3] that is further thought to be powerful, as the synaptic targets in GPi are close to the soma [4], and b) electrophysiological studies that record very similar activities in both nuclei, at rest [5], in directional arm reaching tasks [6] and during eye closures [7]. Furthermore, the BG are supposedly the substrates of a generic selection process among cortical inputs, but the classical explanation of this selection stemming from the segregation between a pathway (with the direct striatal neurons targeting straight the GPi) and an pathway (with the indirect striatal neurons targeting the GPe) is not compatible with primate anatomical data showing a nearly total overlap in the efferences of the striatal neurons. To address these problems, we parameterized mean-field models with the aim of fitting to an unprecedented large body of quantitative data obtained in primates. These data fall into two categories: a) anatomical data that directly constrain the parameter numerical value (they are the mean number of varicosities along the axons or the localization of the synapses along the dendrites) and b) electrophysiological data that can be used to constrain the model in simulation (they are the firing rate recorded either at rest, or when a given neurotransmitter is experimentally blocked by an antagonist). We built two matching fitness functions assessing how models were respecting these data, and we optimized them with the NSGA-2 multi-objective evolutionary algorithm [8]. We obtained more than one thousand different sets of parameters maximizing both fitnesses, and each of these models constitute a different answer to the initial paradox. By specifically examining the GPe -> GPi parameters, we found that a moderate strength of this projection is not antagonist with similar activities in both nuclei. Broader analysis led to the prediction that the GPe and GPi inputs are unbalanced, the former receiving stronger afferences from the striatum and the subthalamic nucleus than the latter. We also tested the selection capability of our plausible models by mimicking a classical arm-reaching task performed in monkeys [6], and we show that (1) the plausible basal ganglia models that we obtained are performing selections, indicating that the mainstream theory of selection is still valid without the postulate of a segregation into and pathways, and that (2) this selection is performed more efficiently with a diffused projection pattern from the GPe into the GPi, as compared to a focused projection.

A Role for Dopamine-Mediated Learning in the Pathophysiology and Treatment of Parkinson’s Disease

Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervation profoundly alters motor performance, as in Parkinson's disease (PD); however, the extent to which these symptoms reflect impaired motor learning is unknown. Here, we demonstrate a D2 receptor blockade-induced aberrant learning that impedes future motor performance when dopamine signaling is restored, an effect diminished by coadministration of adenosine antagonists during blockade. We hypothesize that an inappropriate corticostriatal potentiation in striatopallidal cells of the indirect pathway underlies aberrant learning. We demonstrate synaptic potentiation in striatopallidal neurons induced by D2 blockade and diminished by application of an adenosine antagonist, consistent with behavioral observations. A neurocomputational model of the basal ganglia recapitulates the behavioral pattern and further links aberrant learning to plasticity in the indirect pathway. Thus, D2-mediated aberrant learning may contribute to motor deficits in PD, suggesting new avenues for the development of therapeutics.

What do the basal ganglia do? A modeling perspective

Basal ganglia (BG) constitute a network of seven deep brain nuclei involved in a variety of crucial brain functions including: action selection, action gating, reward based learning, motor preparation, timing, etc. In spite of the immense amount of data available today, researchers continue to wonder how a single deep brain circuit performs such a bewildering range of functions. Computational models of BG have focused on individual functions and fail to give an integrative picture of BG function. A major breakthrough in our understanding of BG function is perhaps the insight that activities of mesencephalic dopaminergic cells represent some form of 'reward' to the organism. This insight enabled application of tools from 'reinforcement learning,' a branch of machine learning, in the study of BG function. Nevertheless, in spite of these bright spots, we are far from the goal of arriving at a comprehensive understanding of these 'mysterious nuclei.' A comprehensive knowledge of BG function has the potential to radically alter treatment and management of a variety of BG-related neurological disorders (Parkinson's disease, Huntington's chorea, etc.) and neuropsychiatric disorders (schizophrenia, obsessive compulsive disorder, etc.) also. In this article, we review the existing modeling literature on BG and hypothesize an integrative picture of the function of these nuclei.

A computational model of basal ganglia and its role in memory retrieval in rewarded visual memory tasks

Visual working memory (WM) tasks involve a network of cortical areas such as inferotemporal, medial temporal and prefrontal cortices. We suggest here to investigate the role of the basal ganglia (BG) in the learning of delayed rewarded tasks through the selective gating of thalamocortical loops. We designed a computational model of the visual loop linking the perirhinal cortex, the BG and the thalamus, biased by sustained representations in prefrontal cortex. This model learns concurrently different delayed rewarded tasks that require to maintain a visual cue and to associate it to itself or to another visual object to obtain reward. The retrieval of visual information is achieved through thalamic stimulation of the perirhinal cortex. The input structure of the BG, the striatum, learns to represent visual information based on its association to reward, while the output structure, the substantia nigra pars reticulata, learns to link striatal representations to the disinhibition of the correct thalamocortical loop. In parallel, a dopaminergic cell learns to associate striatal representations to reward and modulates learning of connections within the BG. The model provides testable predictions about the behavior of several areas during such tasks, while providing a new functional organization of learning within the BG, putting emphasis on the learning of the striatonigral connections as well as the lateral connections within the substantia nigra pars reticulata. It suggests that the learning of visual WM tasks is achieved rapidly in the BG and used as a teacher for feedback connections from prefrontal cortex to posterior cortices.

A computational model of basal ganglia involved in the cognitive control of visual perception

Event Abstract Back to Event A computational model of basal ganglia involved in the cognitive control of visual perception Fred H. Hamker1* and Julien Vitay2 1 Technichal University Chemnitz, Computer Science Department, Germany 2 Westfalischen Wilhelms Universitat, Psychologisches Institut II, Germany Goal-directed visual perception requires to maintain and manipulate a template of the desired target in visual working memory (WM) that allows to bias processing in the posterior lobe. It is still unclear how such goal-directed perception is implemented in the brain. We propose that such interaction between visual WM and attentional processing involves a network of brain areas consisting of inferotemporal cortex (IT) for the retrieval of visual information associated to the target, dorsolateral-prefrontal cortex (dlPFC) for the manipulation and maintenance of the target template in face of distractors, medial temporal lobe (MTL) regions for rapid encoding and novelty detection, as well as basal ganglia (BG) for the switching and activation of WM representations in dlPFC.We designed a novel computational model of BG, while it interacts with a model of perirhinal cortex (PRh, part of the medial temporal lobe) and a simple model of dlPFC for memorizing objects, to explore how the BG might be involved in the cognitive control of visual perception. The BG model is composed of the striatum receiving connections from PRh and dlPFC. The striatum inhibits SNr which in turn tonically inhibits a thalamic nucleus interacting with PRh. Homeostatic Hebbian learning takes place simultaneously in the connections from cortical areas to the striatum (representing the context of of the task) and in the connections from striatum to SNr as well as within SNr (learning to retrieve the correct representation). Moreover, a dopaminergic cell learns to compute the difference between the reward actually received and the expectation of reward based on striatal representations and modulates learning in the other areas.We applied this model to simultaneously learn delayed matching-to-sample (DMS) and delayed nonmatching-to-sample (DNMS) tasks. Whether DMS or DNMS should be performed is defined by a contextual information presented after the sample and before the search array composed of two targets. Reward is given to the system when it selects the correct target through thalamic stimulation of PRh. The model has to represent efficiently the context in the striatum to solve the tasks and is able to learn them concurrently after 400 trials, independently of the number of cells in SNr, what denotes a parallel search of the correct representation. If at the beginning of learning, several cells in SNr can become selective for the same striatal pattern, the learned competition between them progressively selects the only one that disinhibits the correct target. The reward-predictive value of striatal representations also takes into account the probability of reward associated to an object.Similarly to the PVLV model of O'Reilly and Frank, our model reproduces the reward-related firing pattern of dopaminergic cells in conditioning. However, our model highlights the role of BG processing in visual WM processes, not only in its cognitive component but also in the retrieval of target information. It explains how vision can be guided by the goals of the task at hand. Conference: Bernstein Conference on Computational Neuroscience, Frankfurt am Main, Germany, 30 Sep - 2 Oct, 2009. Presentation Type: Poster Presentation Topic: Decision, control and reward Citation: Hamker FH and Vitay J (2009). A computational model of basal ganglia involved in the cognitive control of visual perception. Front. Comput. Neurosci. Conference Abstract: Bernstein Conference on Computational Neuroscience. doi: 10.3389/conf.neuro.10.2009.14.006 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 25 Aug 2009; Published Online: 25 Aug 2009. * Correspondence: Fred H Hamker, Technichal University Chemnitz, Computer Science Department, Chemnitz, Germany, fred.hamker@informatik.tu-chemnitz.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Fred H Hamker Julien Vitay Google Fred H Hamker Julien Vitay Google Scholar Fred H Hamker Julien Vitay PubMed Fred H Hamker Julien Vitay Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

THE ROLE OF THE BASAL GANGLIA IN EXPLORATION IN A NEURAL MODEL BASED ON REINFORCEMENT LEARNING

We present a computational model of basal ganglia as a key player in exploratory behavior. The model describes exploration of a virtual rat in a simulated water pool experiment. The virtual rat is trained using a reward-based or reinforcement learning paradigm which requires units with stochastic behavior for exploration of the system's state space. We model the Subthalamic Nucleus-Globus Pallidus externa (STN-GPe) segment of the basal ganglia as a pair of neuronal layers with oscillatory dynamics, exhibiting a variety of dynamic regimes such as chaos, traveling waves and clustering. Invoking the property of chaotic systems to explore state-space, we suggest that the complex exploratory dynamics of STN-GPe system in conjunction with dopamine-based reward signaling from the Substantia Nigra pars compacta (SNc) present the two key ingredients of a reinforcement learning system.