Deep reinforcement learning-based impact time control guidance law with constraints on the field-of-view

Abstract

This paper proposes a novel three-dimensional (3D) intelligent impact time control guidance (ITCG) law with the field-of-view (FOV) strictly constrained based on nonlinear relative motion relationship. This novel guidance law can be utilized to coordinate multiple missiles to attack a target simultaneously. Firstly, the precise time-to-go estimation approach of the 3D coupling proportional navigation guidance (PNG) law is derived, which can improve accuracy of impact time. Secondly, a modified time biased term c⁎ including the guidance gain k1 which can accelerate the convergence rate is incorporated into 3D-PNG by feeding back the FOV error. Therefore, the optimal trajectory can be got to achieve precise impact time, in the meantime, the FOV constraint can be satisfied strictly. Then, the guidance gain k1 is obtained by deep deterministic policy gradient (DDPG) in the reinforcement learning (RL) framework, which makes the proposed guidance law meet smaller time error and has less energy loss. Furthermore, the proposed intelligent law can obtain optimal guidance gains quickly and avoid the tedious manual setting. Ultimately, simulation results demonstrate that missiles can impact the target precisely at desired impact times without violating FOV limit and the optimal guidance gain k1 is obtained adaptively and quickly in various initial conditions.