The rate of convergence is a vital factor in determining the outcome of the mission execution of unmanned aerial vehicle (UAV) swarms. However, the difficulty of developing a rapid convergence strategy increases dramatically with the growth of swarm scale. In the present work, a novel fractional-order flocking algorithm (FOFA) is proposed for large-scale UAV swarms. First, based on the interaction rules of repulsion, attraction and alignment among swarm individuals, fractional calculus is introduced to replace traditional integer-order velocity updating, which enables UAVs to utilize historical information during flight. Subsequently, the convergence of the algorithm is theoretically analyzed. Some sufficient convergence conditions for the FOFA are presented by exploiting graph theory. Finally, the simulation results validate that our proposed FOFA performs much better than traditional flocking algorithms in terms of convergence rate. Meanwhile, the relationships between the fractional order of the FOFA and the convergence time of the UAV swarm are discussed. We find that under certain conditions, the fractional order is strongly correlated with the convergence rate of the UAV swarm; that is, a small fractional order (more consideration of historical information) leads to better performance. Moreover, the fractional order can be used as an important parameter to control the convergence rate of a large-scale UAV swarm.