Reinforcement Learning in non-stationary environments: An intrinsically motivated stress based memory retrieval performance (SBMRP) model

Abstract

Biological systems are said to learn from both intrinsic and extrinsic motivations. Extrinsic motivations, largely based on environmental conditions, have been well explored by Reinforcement Learning (RL) methods. Less explored, and more interesting in our opinion, are the possible intrinsic motivations that may drive a learning agent. In this paper we explore such a possibility. We develop a novel intrinsic motivation model which is based on the well known Yerkes and Dodson stress curve theory and the biological principles associated with stress. We use a stress feedback loop to affect the agent's memory capacity for retrieval. The stress and memory signals are fed into a fuzzy logic system which decides upon the best action for the agent to perform against the current best action policy. Our simulated results show that our model significantly improves upon agent learning performance and stability when objectively compared against existing state-of-the-art RL approaches in non-stationary environments and can effectively deal with significantly larger problem domains.