Human-Inspired Meta-Reinforcement Learning Using Bayesian Knowledge and Enhanced Deep Q-Network

Abstract

Over the last decades, there has been growing interest in research in multiple and interdisciplinary fields of human-AI computing. In particular, approaches integrating human’s perspective and design with reinforcement learning (RL) have received more attention. However, the current research on RL may need to consider its enhancement from human-inspired approaches further. In this work, we focus on enabling a meta-reinforcement learning (meta-RL) agent to achieve adaptation and generalization, according to modeling Markov Decision Processes (MDP) using Bayesian knowledge and analysis. By introducing a novel framework called human-inspired meta-RL (HMRL), we incorporate the agent performing resilient actions to leverage the dynamic dense reward based on the knowledge and prediction of a Bayesian analysis. The proposed framework can make the agent learn generalization and prevent the agent from failing catastrophically. The experimental results show that our approach helps the agent reduce computational costs with learning adaptation. In addition to the system design, we have also extended further algorithmic improvement based on learning within a deep Q-network (DQN) implementations for more complicated future tasks, which compared replay buffers to possibly enhance the optimization process. Finally, we conclude and anticipate that integrating human-inspired meta-RL can enable learning more formulations relating to robustness and scalability, leading to promising directions and more complex AI goals in the future.