Deep adaptive group-based input normalization for financial trading

Abstract

Deep Reinforcement Learning (DRL) is among the state-of-the-art approaches for training agents for decision-making problems, such as financial trading. However, training DRL agents for such tasks is not always straightforward, since the noisy and non-stationary nature of financial data aggravate the already unstable training of DRL models. As a result, using DRL methods for such tasks require devoting significant effort for hyper-parameter tuning, as well as for designing the appropriate input pre-processing schemes. The latter is especially important, given the non-stationary nature of financial data, along with the ability of DRL agents to easily overfit the data, limiting their generalization abilities. In this work, we propose overcoming these limitations by introducing a differentiable, parameterized normalization scheme that allows for learning how the data should be normalized, along with the DRL model. More specifically, we propose dynamically normalizing the input according to various time-series statistics, which allows for adapting the model on-the-fly to the current mode of the data. At the same time, employing a segmentation scheme for extracting the statistics of the data allows for better capturing the variations of the input time-series and leading to more stationary representations. The proposed method is formulated as a series of neural layers that can be efficiently implemented using virtually any DL framework. The effectiveness of the proposed method against various normalization approaches is validated using two FOREX datasets and a state-of-the-art policy-based DRL approach.