Schizophrenia: A Computational Reinforcement Learning Perspective

Abstract

As one of the most complex neurocognitive disorders, schizophrenia (SZ) is a devastating condition for which the underlying sources are far from being fully understood. Indeed, it is likely that there are multiple etiologies to the disease and heterogeneity within the population. Moreover, it is impossible to understand from a purely mechanistic basis how a patient would come to believe so strongly in delusions as to, for example, gouge out his own eyes. Nevertheless, science marches forward, and the last 30 years or so have produced a wealth of knowledge regarding some of the risk factors, genetics, pharmacology, cognitive deficits, and underlying neurobiology associated with the disease. In part because of the efficacy of antipsychotic treatments via dopamine D2 receptor blockade, the majority of this research focuses on dysfunctions of the dopaminergic system, in both frontal cortex and basal ganglia, thought to be related to negative and positive symptoms, respectively. At the neurocognitive level,muchof the focushasbeenondysfunctionwithin dorsolateralprefrontal cortical circuits and their contributions to working memory, cognitive control, and attentional shifting. While dopamine plays a critical role in all these processes, it is perhaps more centrally related to aspects of motivational processing, which is surprisingly understudied in SZ. Indeed, it is possible to account for many of the frontal-dependent cognitive deficits in SZbypositingamore coredeficit in themotivational ‘‘gating’’ system for determining which information patients should ‘‘care’’ about and what they might ignore. Given the complexity of neural circuits involved in both cognitive and motivational functions, it becomes dauntingly difficult to capture the possible interactions of these circuits, and particularly how they are disrupted in SZ, with simple verbal depictions and static anatomical diagrams. Here I consider the potential application of computational neural network models as a principled and dynamic tool for exploring these interactions and psychopathology associated with dopaminergic dysfunction in SZ and which can lead to new testable predictions at both the neural and behavioral levels. These models enable one to simulate various anatomical and physiological pieces of data, using mathematical equations that capture how groups of neurons communicate activity to other neurons within and between brain areas. By incorporating aspects of neuronal physiology, connectivity, and synaptic plasticity within the basal ganglia–frontal cortical system, one can examine dynamics of this circuitry and how it may go awry. At the same time, it is not tractable to try to incorporate every known biological detail into a model, particularly when the goal is to discover how an entire system of brain regions interact to produce behavior. Thus, the models are also constrained by the need to account for existing data at these higher levels, such as effects of focal lesions or pharmacological manipulation on behavior. Critically, the models make new predictions about how the system works that would likely not have emerged otherwise and often were not conceived by the modeler prior to being built. Models can then be tested and refined and their implications explored in neurological conditions. To sum up a large body of basic research, models of frontostriatal function have generally suggested that these circuits support the following—(1) action selection: as in when making a choice among multiple competing alternatives and (2) reinforcement learning: as in modifying expectations and behavior following positive and negative outcomes. For the former process, ‘‘actions’’ to be selected include both lower level motor programs, consistent with the traditionally ascribed role of the basal ganglia in motor control, and higher level cognitive actions, such as when and when not to update/manipulate the contents of working memory. Reinforcement learning then operates on these actions such that adaptive actions are more likely to be repeated, whereas maladaptive actions are suppressed. Critically, according to both the models and available electrophysiological evidence, positive outcomes are reflected in terms of deviations from current expectations, a term referred to as a ‘‘positive prediction error,’’ and are encoded by phasic bursts of dopamine. Similarly, negative prediction errors are encoded by phasic dips or pauses in dopaminergic activity. These phasic bursts and dips modify corticostriatal synaptic plasticity, allowing the system To whom correspondence should be addressed; tel: 520-626-4787, fax: 520-621-9306, e-mail: mfrank@u.arizona.edu. Schizophrenia Bulletin doi:10.1093/schbul/sbn123