A multi-agent convolution deep reinforcement learning network for aeroengine fleet maintenance strategy optimization

Abstract

The core task of aeroengine fleet maintenance strategy optimization is to realize the collaborative optimization of each engine and fleet resources. However, the existing maintenance strategy method cannot precisely coordinate the maintenance plan of each engine in the fleet, especially when the state space dimension of the fleet is high and the maintenance action is complex. To solve these problems, a multi-agent convolution deep reinforcement learning network, which is a variant of the traditional deep Q-learning network, is proposed to accurately optimize the maintenance strategy of engine fleet. Firstly, the convolutional deep learning network is used as the Q-value computing network in the deep Q-learning method to extract the low-dimensional features that can better represent the high-dimensional state of the engine fleet. Secondly, in order to effectively make decisions on the complex maintenance actions of each engine, the single-agent in the deep Q-learning method is extended to a multi-agent structure. Each agent uses an independent Q-value calculation network and combined with constraints to make decisions only on the maintenance actions of its corresponding engine. At the same time, reinforcement learning mechanism is used to complete the collaborative optimization of engine maintenance strategies. Finally, through two groups of simulation experiments, the rough optimization and fine optimization of fleet maintenance strategy are realized respectively, and compared with some extensive optimization methods. The experimental results show the superiority of the proposed method in aeroengine fleet maintenance strategy optimization.