Grain boundary (GB) engineering includes grain size and GB segregation. Grain size has been proven to affect the electrical properties of Mg3(Sb, Bi)2 at low temperatures. However, the formation mechanism of GB segregation and what kind of GB segregation is beneficial to the performance are still unclear. Here, the Ga/Bi cosegregation at GBs and Mg segregation within grains optimize the transport of electrons and phonons simultaneously. Ga/Bi cosegregation promotes the formation of Janus-like structures due to the diverse ordering tendencies of liquid Mg3Sb2 and Mg3Bi2 and the absence of a solid solution of Ga/Bi. The Janus-like structure significantly reduces the room-temperature lattice thermal conductivity by introducing diverse microdefects. Meanwhile, a coherent interface between the nano Mg segregation region and the matrix is formed, which reduces the thermal conductivity without affecting the carrier transport. Furthermore, the band structure calculations show that Ga doping introduces the resonance level, increasing the Seebeck coefficient. Finally, the lattice thermal conductivity reaches ∼0.4 W m-1 K-1, and a high average ZT of 1.21 between 323 and ∼773 K is achieved for Mg3.2Y0.02Ga0.03Sb1.5Bi0.5. This work provides guidance for improving the thermoelectric performance via designing cosegregation.