Recently, the search for materials with high photoelectric conversion efficiency has emerged as a significant research hotspot. Unlike p-n junctions, the bulk photovoltaic effect (BPVE) can also materialize within pure crystals. Here, we propose wurtzite and zinc blende semiconductors without inversion symmetry (AgI, GaAs, CdSe, CdTe, SiGe, ZnSe, and ZnTe) as candidates for achieving the BPVE and investigate the factors that affect the shift current. GaAs with a wurtzite structure exhibits the highest shift current value of 31.8 μA V-2 when spin-orbit coupling is considered. Meanwhile, the peak position of the maximum linear optical conductivity and shift current in the wurtzite structure is lower than that in the zinc blende structure. In addition, we also found that strong covalency within the same main axis group element significantly influences the shift current, exemplified by wurtzite SiGe, which exhibits 15.8 μA V-2. Our research highlights the importance of a smaller band gap, reduced carrier effective mass, and increased covalency in achieving a substantial shift current response. Ultimately, this study provides valuable insights into the interplay of the structural and electronic properties, offering directions for the discovery and design of materials with an enhanced BPVE.