Deep Recurrent Q-Learning for Research on Complex Economic System

Abstract

Complex economic system theory studies the socio-economic system in a dynamic, complex ideas. It is a more comprehensive economic theory established in non-equilibrium basis. At present, under the guidance of complex economic system theory, multi-agent based modeling and simulation is a universal research method to study economic system. However, the agent's behavior usually is fixed according to the simulation software's setting, which is difficult to explain the phenomena of feedback and emergence in the economic system. To address this problem, we first design a simulation economic system. In the system, there are two types of abstract micro economic agents, bank and enterprise. An improved version of Deep Q-Network (DQN), Deep Recurrent Q-Network (DRQN) was introduced to handle the decision-making of micro economic agents. Finally, we studied the observed behavior of agents in the experiment. The experimental results show that the behavior of economic agents can actively adapt to changes in the system environment and continue to be optimized with the accumulation of experience. Because of taking non-fixed decision-making strategies, this method can reflect the phenomena of internal feedback and emergence in economic system. Therefore, simulation based on DRQN is a simulation method that can better reflect agent decision-making in real economic system and it can play an important role in the study of complex economic system.