Learning with an asymmetric teacher: Asymmetric dopamine-like response can be used as an error signal for reinforcement learning

Abstract

Event Abstract Back to Event Learning with an asymmetric teacher: Asymmetric dopamine-like response can be used as an error signal for reinforcement learning Rea Mitelman1*, Mati Joshua1 and Hagai Bergman1 1 Hebrew University of Jerusalem, Israel According to many computational models, dopamine (DA) neurons in the basal ganglia (BG) play a major role in reinforcement learning. DA signal is proportional to the difference between actual and predicted reward, hence it could serve as the error signal in a temporal difference (TD) learning algorithm implemented in the BG. Indeed, a proportional increase in the firing rate of DA neurons in states with higher values than expected (the positive domain of the error signal) has been found experimentally. However, many studies indicate that DA neurons do not decrease their firing rate symmetrically in the negative domain. Some studies report a smaller gain relative to the positive domain, whereas others report a decrease in the firing to a constant level. Our work focuses on using such an asymmetric error signal in a TD-like computational algorithm. We simulated a probabilistic classical conditioning task in which the agent sequentially received stimuli with different probabilities of reward or aversion. The algorithm calculated the value of each of the stimuli, using an asymmetric TD signal. This was done by manipulating the negative domain of the error signal function by decreasing its slope by a multiplicative factor smaller than one, fixing its negative values to a constant negative level, or fixing them to zero. We show that learning can be achieved when the negative domain of the error signal function is either constant negative or with reduced gain, although it is slower than with a symmetric error signal. However, learning cannot be achieved with a constant value of zero for the negative domain of the error signal function. We examined learning by comparing the values the algorithm assigned to the stimuli with those assigned by a symmetric TD algorithm. These values had a non-linear concave trend, with higher calculated values than those assigned by the symmetric TD algorithm. This suggests that the DA asymmetric signal could be used as an error signal in a TD algorithm as implemented in the BG and that DA asymmetric coding does not require a complementary BG modulatory error signal. We further hypothesize that the concave values of the error signal curve could thus lead to some aspects of irrational human behavior. Conference: Bernstein Symposium 2008, Munich, Germany, 8 Oct - 10 Oct, 2008. Presentation Type: Poster Presentation Topic: All Abstracts Citation: Mitelman R, Joshua M and Bergman H (2008). Learning with an asymmetric teacher: Asymmetric dopamine-like response can be used as an error signal for reinforcement learning. Front. Comput. Neurosci. Conference Abstract: Bernstein Symposium 2008. doi: 10.3389/conf.neuro.10.2008.01.089 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 17 Nov 2008; Published Online: 17 Nov 2008. * Correspondence: Rea Mitelman, Hebrew University of Jerusalem, Jerusalem, Israel, ream@alice.nc.huji.ac.il Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Rea Mitelman Mati Joshua Hagai Bergman Google Rea Mitelman Mati Joshua Hagai Bergman Google Scholar Rea Mitelman Mati Joshua Hagai Bergman PubMed Rea Mitelman Mati Joshua Hagai Bergman Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.