Aiming at the problem of Unmanned Aerial Vehicle(UAV) formation path planning under complex constraints, a UAV formation path planning method based on the combination of Rapidly exploring Random Tree (RRT) and Rauch-Tung-Striebel (RTS) filter is proposed. Firstly, a path planning algorithm based on the improved RRT algorithm with adaptive step size is de-signed to solve the problem that the RRT algorithm is easy to fall into local optimum. Then, an RTS filter is introduced to smooth the trajectory planned by the improved RRT algorithm to achieve curvature continuity. Finally, taking the smooth trajectory as the reference, a UAV formation path planning algorithm over the Artificial Potential Field (APF) method is designed. The simulation results show that the designed UAV formation path planning algorithm can solve the planning problems of single trajectory and formation trajectories in complex constrained space, and can plan the formation trajectory with continuous curvature, to facilitate the UAV trajectory tracking control.