MDAEN: Multi-Dimensional Attention-based Ensemble Network in Deep Reinforcement Learning Framework for Portfolio Management

Abstract

Reinforcement Learning algorithms are widely applied in many diverse fields, including portfolio management. Ensemble of Identical Independent Evaluators (EIIE) framework proposed by Jiang et al. achieved portfolio management based on their deep reinforcement learning algorithm. In the implementation of EIIE framework, a neural network such as the Convolutional Neural Network is applied as the policy network, to uncover more patterns in the data. However, this network typology is inefficient due to its simple structure. To overcome the shortcoming of EIIE framework, this paper introduces a novel algorithm, the Multi-Dimensional Attention-based Ensemble Network (MDAEN) strategy, which consists of a features-attention module and an assets-attention module. The MDAEN applies different types of attention mechanisms to extract information from the assets. Having adopted the reinforcement learning framework from Jiang et al., the agent is able to process transactions through MDAEN in a market. In our portfolio establishment, Bitcoin together with eleven other cryptocurrencies is selected to validate the performance of MDAEN against seven traditional portfolio strategies and EIIE. The experimental result demonstrates the efficacy of our strategy outperforming all other strategies by at least 35% in profitability and at least 30% in Sharpe Ratio.