Defense penetration strategy for unmanned surface vehicle based on modified soft actor–critic

Abstract

This paper investigates the defense penetration problem for unmanned surface vehicle (USV) with limited detection range under unknown defenders. A novel end-to-end defense penetration strategy is proposed for the USV based on modified soft actor–critic (MSAC). Specifically, the state–action space of the attacker is designed by combining the maneuvering model of USV and detection information. To guide the USV to reach the target area under the interception of defenders, the penetration reward is constructed. Then, the long short-term memory network (LSTM) is applied in the actor–critic network design. A dynamic training mechanism is proposed by combining curriculum learning and multi-memory pools, which aims to improve the success rate of defense penetration. Finally, simulation results validate the effectiveness and superiority of the proposed defense penetration method for the USV suffering from unknown defenders. The higher success rate can be guaranteed by using MSAC compared to the previous methods.