In this paper, a new derived method is developed for a known numerical differential formula of the Caputo fractional derivative of order $$\gamma \in (1,2)$$ (Li and Zeng in Numerical methods for fractional calculus. Chapman & Hall/CRC numerical analysis and scientific computing, CRC Press, Boca Raton, 2015) by means of the quadratic interpolation polynomials, and a concise expression of the truncation error is given. This new method will be called as the H2N2 method because of the application of the quadratic Hermite and Newton interpolation polynomials. A finite difference scheme with a second order accuracy in space and a $$(3-\gamma )$$-th order accuracy in time based on the H2N2 method is constructed for the initial boundary value problem of time-fractional wave equations. The stability and convergence of the difference scheme are proved. Furthermore, in order to increase computational efficiency, using the sum-of-exponentials to approximate the kernel $$t^{1-\gamma }$$, a fast difference scheme is presented. The problem with weak regularity at the initial time is also discussed with the help of the graded meshes. At each time level, the difference scheme is solved with a fast Poisson solver. Numerical results show the effectiveness of the two difference schemes and confirm our theoretical analysis.