Deep reinforcement learning based on transformer and U-Net framework for stock trading

Abstract

An effective stock-trading strategy offers investors as much profit and as little risk as possible. Capturing volatility trends from historical stock prices and determining trading strategies is extremely challenging. This study proposes an end-to-end model called DRL-UTrans for learning a single stock trading strategy that combines deep reinforcement learning, transformer layers, and a U-Net architecture. In particular, the transformer layer captures complex dynamic patterns in financial markets. The model structure based on the U-Net architecture contains multiple skip connections used to combine long- and short-term features. The input of the model is a windowed stock price sequence, and the output consists of a trading action and action weight. The benefit of having two outputs is that the agent can control the share of buys and sells to reduce investment risk. In addition, a reward function that is sensitive to market volatility is proposed to feed back the market state. Finally, trading data for 10 stocks is extracted from a real financial market to validate the proposed model. The results show that DRL-UTrans has a higher profitability compared with the seven baseline approaches; further, it is effective in sensing market volatility and hedging market risk when encountering stock crashes.