Amygdala and ventral striatum population codes implement multiple learning rates for reinforcement learning

Abstract

Standard models of reinforcement learning in the brain assume that dopamine codes reward prediction errors, and these reward prediction errors are integrated by the striatum to generate state and action value estimates. Recent research suggests that the amygdala also plays a key role in this process, and that the amygdala and striatum learn on different time scales. Here we show that the amygdala, which learns with a faster learning rate, is most effective in lower noise environments where the underlying reward function may be changing on relatively fast time scales. The striatum, on the other hand, has a slower learning rate and therefore is most effective in higher noise environments that change on relatively slow time scales. Having multiple neural systems that learn on different time scales gives the brain an advantage across diverse environments where levels of noise and reward function dynamics may differ.