A multi-agent deep reinforcement learning framework for algorithmic trading in financial markets

Abstract

Algorithmic trading based on machine learning is a developing and promising field of research. Financial markets have a complex, uncertain, and dynamic nature, making them challenging for trading. Some financial theories, such as the fractal market hypothesis, believe that the markets behave based on the collective psychology of investors who trade with different investment horizons and interpretations of information. Accordingly, a multi-agent deep reinforcement learning framework is proposed in this paper to trade on the collective intelligence of multiple agents, each of which is an expert trader on a specific timeframe. The proposed framework works in a hierarchical structure in which the flow of knowledge is from the agents trading at higher timeframes to the agents trading at lower timeframes, making them highly robust to the noise in financial time series. The Deep Q-learning algorithm is utilized for training the agents in the framework. The performance of the proposed framework is evaluated through extensive numerical experiments conducted on a historical dataset of the EUR/USD currency pair. The results demonstrate that the proposed multi-agent framework, based on several return-based and risk-based performance measures, outperforms single independent agents and several benchmark trading strategies in all investigated trading timeframes. The robust performance of the multi-agent framework throughout the trading period makes it suitable for algorithmic trading in financial markets.