Algorithmic trading using combinational rule vector and deep reinforcement learning

Abstract

Algorithmic trading rules are widely used in financial markets as technical analysis tools for security trading. However, traditional trading rules are not sufficient to make a trading decision. In this paper, we propose a new algorithmic trading method called CR-DQN, which incorporates deep Q-learning with two popular trading rules: moving average (MA) and trading range break-out (TRB). The input of deep Q-learning is combinational rule vectors, whose component is a linear combination of 140 rules produced by MA and TRB with different parameters. Due to non-stationary characteristics, we devise a reward driven combination weight updating scheme to generate combinational rule vectors, which can capture intrinsic features of financial data. Since the sparse reward exists in CR-DQN, we design a piecewise reward function which shows great potential in the experiments. Taking combinational rule vectors as input, the LSTM based Deep Q-learning network is used to learn an optimal algorithmic trading strategy. We apply our model to both Chinese and non-Chinese stock markets, and CR-DQN exhibits the best performance on a variety of evaluation criteria compared to many other approaches, demonstrating the effectiveness of the proposed method.