Phononic band gap crystals (PnCs) are periodic composite materials and well known for their novel property that can prohibit the propagation of mechanical waves in certain range of frequency. This paper develops the finite element method to calculate band structures of bi-material phononic crystals. Through finite element analysis, complete band gap for longitudinal and transverse waves are obtained by characterizing the dispersion relation in phononic crystals. Phononic crystals with different inclusion shapes in a square and hexagonal unit cell are investigated to study the influence of unit cell topology on band gap size. For a specific pattern, the existence of complete band gap in relation to the density and Lamé constant modulus of composites is studied and critical density ratio and Lamé constant ratio of inclusions versus base material for opening complete band gap are given. The results provide theoretical guidance for designing phononic crystals in practical applications.