Abstract To address the navigation challenges of mobile robot swarm coordination in complex environments, this paper presents a distributed navigation approach for mobile robot swarm. The study initially formulates the problem of robot swarm navigation as a partially observable Markov decision process, employing a distributed proximal policy optimization algorithm to train decision models mapping observations to actions directly. Furthermore, it adopts imitation learning pre-training to maximize expert decisions in the initial training phase, thereby reducing exploration space. Lastly, an improved path point generation algorithm along with a spatiotemporal reachability calculation method is proposed to guide the Deep Reinforcement Learning policy in accomplishing long-distance indoor navigation. Comprehensive performance evaluations are conducted in simulated environments to compare the proposed method with existing approaches. Results demonstrate that the proposed approach effectively enhances both convergence speed and model performance, while significantly improving the navigation capabilities of robot swarms in complex environments.