In this paper, the origins of spurious solutions occurring in the high-order finite difference methods are studied. Based on a uniform mesh, spurious modes are found in the high-order one-sided finite difference discretizations of many eigenvalue problems. Spurious modes are classified as spectral pollution and non-spectral pollution. The latter can be partially avoided by mesh refinement, while the former persists when the mesh is refined. Through numerical studies of some prototype eigenvalue problems, such as those of the Helmholtz and beam equations, we show that perfect central differentiation schemes do not produce any spurious modes. Nevertheless, high-order central difference schemes encounter difficulties in implementing complex boundary conditions. We further show that the central difference schemes will produce spurious modes as well, if asymmetric approximation is involved in boundary treatments. In general, central difference schemes are less likely to produce spurious modes than one-sided difference schemes.