Algorithmic trading rules are widely used in financial markets as technical analysis tools for security trading. However, traditional trading rules are not sufficient to make a trading decision. In this paper, we propose a new algorithmic trading method called CR-DQN, which incorporates deep Q-learning with two popular trading rules: moving average (MA) and trading range break-out (TRB). The input of deep Q-learning is combinational rule vectors, whose component is a linear combination of 140 rules produced by MA and TRB with different parameters. Due to non-stationary characteristics, we devise a reward driven combination weight updating scheme to generate combinational rule vectors, which can capture intrinsic features of financial data. Since the sparse reward exists in CR-DQN, we design a piecewise reward function which shows great potential in the experiments. Taking combinational rule vectors as input, the LSTM based Deep Q-learning network is used to learn an optimal algorithmic trading strategy. We apply our model to both Chinese and non-Chinese stock markets, and CR-DQN exhibits the best performance on a variety of evaluation criteria compared to many other approaches, demonstrating the effectiveness of the proposed method.
Read full abstract