The present work proposes a novel modified smoothed finite element method (M-SFEM) to calculate the band structures of two-dimensional in-plane elastic waves in phononic crystals (PCs). Using the gradient smoothing technique over the cell-based smoothing domains, the cell-based gradient smoothing operation can offer ‘proper softening effects’ in SFEM modeling. According to the generalized integration rules, by simply shifting integration points to an unconventional location in the mass matrix, the accuracy of the M-SFEM model can be further improved. Numerical examples are presented for PCs using the proposed method for the computation of band gap (BG) frequency regions. The accuracy and efficiency of the modified SFEM are compared with those of the corresponding FEM and SFEM. The advantages of the modified SFEM for computing the BGs in PCs are discussed as compared to the conventional FEM. The results show the performance improvement of the M-SFEM compared to the FEM and SFEM.