AbstractBy the sum‐of‐exponentials approximation and quadric interpolation, a fast (3 − α)‐order numerical formula has been presented to approximate the Caputo fractional derivative. With the use of the formula, an efficient difference method has been proposed for solving the time‐fractional diffusion equation (TFDE), which allows recursive computation and can significantly reduce the storage and computational cost. The stability and convergence of the difference method have been studied by the discrete energy method, and it is proved that the method can achieve (3 − α)‐order accuracy in time and second‐order accuracy in space. Furthermore, to deal with the TFDE with some weak singularities at the origin, a fast algorithm on graded meshes has been presented. Numerical examples verify the theoretical prediction and illustrate the efficiency of the schemes.