A deep Q-learning based algorithmic trading system for commodity futures markets

Abstract

Nowadays, investors seek more sophisticated decision-making tools that maximize their profit from investing in the financial markets by suitably determining the optimal position, trading time, price, and volume. This paper proposes a novel intraday algorithmic trading system for volatile commodity futures markets based on a Deep Q-network (DQN) algorithm and its robust double-version (DDQN). The higher volatility, leverage property, and more liquidity in futures contracts give investors more opportunity to take advantage of speculative behaviors with a relatively small amount of capital; however, the volatility brings more difficulties in the learning phase. As an essential prerequisite to training and evaluating any trading algorithm in the futures market, we develop a simulator to replicate a real futures exchange market environment that executes recommended trading signals by handling the clearing and margin management and the pre-order checking mechanisms. Moreover, this study provides a new definition of the continuous state and action spaces that match the futures market's characteristics. To address the curse of dimensionality, we utilize a multi-agent architecture equipped with the Gated Recurrent Unit (GRU) networks to approximate the Q-values functions. The experimental results demonstrate that implementing the proposed trading algorithms (especially the DDQN) into the actual intraday data of gold coin futures contracts significantly outperforms the benchmarks in terms of return, risk, and risk-adjusted return.