Twin-system recurrent reinforcement learning for optimizing portfolio strategy

Abstract

Portfolio management is important for sequential investment decisions in response to fluctuating financial markets. As portfolio management can be formulated as a sequential decision-making problem, it has been addressed using reinforcement learning in recent years. However, reinforcement learning methods face challenges in addressing portfolio management problems considering practical constraints. To overcome the limitations, this study proposes a twin-system approach, establishing a tractable twin that mirrors the original problem but with more manageable constraints and system dynamics. Once an optimized portfolio strategy is achieved within the tractable twin, the proposed mapping function translates it back to the original problem, ensuring the retention of optimized performance. Unlike the previous study, the proposed recurrent reinforcement learning method optimizes the portfolio strategy for a single agent managing all candidate assets. This method allows for comprehensive investment decisions by incorporating the features of candidate assets, leading to a more globally optimized portfolio strategy. Experimental studies demonstrate that the proposed method consistently outperforms benchmark strategies on the US sector and foreign exchange portfolios.