Dopamine, reward conditioning, and robot behavior

Abstract

In this paper, we explore the connection between robot learning and a computational model of a neuromodulatory system. Our model is based on a set of anatomical and physiological properties of the mammalian dopamine system, one of several diffuse ascending systems of the brain known to play a role in learning and plasticity. In the model, the output of the dopamine system acts as a value signal, which gates synaptic changes in sensory and motor areas. As is observed in animal experiments, the neuromodulatory system exhibits characteristic patterns of change during reward conditioning. Different sets of neural units generate precisely timed signals that exert positive and negative effects on neuroplasticity. When the robot is exposed to different environmental conditions, we observe changes in the development of neural connections within the neuromodulatory system that depend on the robot's interaction with the environment.