Finite difference method as a popular numerical method has been widely used to solve fractional diffusion equations. In the general spatial error analyses, an assumption $$u\in C^{4}({\bar{\varOmega }})$$ is needed to preserve $${\mathcal {O}}(h^{2})$$ convergence when using central finite difference scheme to solve fractional sub-diffusion equation with Laplace operator, but this assumption is somewhat strong, where u is the exact solution and h is the mesh size. In this paper, a novel analysis technique is proposed to show that the spatial convergence rate can reach $${\mathcal {O}}(h^{\min (\sigma +\frac{1}{2}-\epsilon ,2)})$$ in both $$l^{2}$$ -norm and $$l^{\infty }$$ -norm in one-dimensional domain when the initial value and source term are both in $${\dot{H}}^{\sigma }(\varOmega )$$ but without any regularity assumption on the exact solution, where $$\sigma \ge 0$$ and $$\epsilon >0$$ being arbitrarily small. After making slight modifications on the scheme, acting on the initial value and source term, the spatial convergence rate can be improved to $${\mathcal {O}}(h^{2})$$ in $$l^{2}$$ -norm and $${\mathcal {O}}(h^{\min (\sigma +\frac{3}{2}-\epsilon ,2)})$$ in $$l^{\infty }$$ -norm. It is worth mentioning that our spatial error analysis is applicable to high dimensional cube domain by using the properties of tensor product. Moreover, two kinds of averaged schemes are provided to approximate the Riemann–Liouville fractional derivative, and $${\mathcal {O}}(\tau ^{2})$$ convergence is obtained for all $$\alpha \in (0,1)$$ . Finally, some numerical experiments verify the effectiveness of the built theory.