A spectral element method (SEM) is proposed for the accurate calculation of band structures of two-dimensional anisotropic photonic crystals. It uses Gauss-Lobatto-Legendre polynomials as the basis functions in the finite-element framework with curvilinear quadrilateral elements. Coordination mapping is introduced to make the curved quadrilateral elements conformal with the problem geometry. Mixed order basis functions are used in the vector SEM for full vector calculation. The numerical convergence speed of the method is investigated with both square and triangular lattices, and with isotropic and in-plane anisotropic media. It is shown that this method has spectral accuracy, i.e., the numerical error decreases exponentially with the order of basis functions. With only four points per wavelength, the SEM can achieve a numerical error smaller than 0.1%. The full vector calculation method can suppress all spurious modes with nonzero eigenvalues, thus making it easy to filter out real modes. It is thus demonstrated that the SEM is an efficient alternative method for accurate determination of band structures of two-dimensional photonic crystals.