The different masses, ionic radii, and chemical valences of the nonequivalent crystallographic sites of thermoelectric (TE) compounds provide an effective way to modulate the thermoelectric performance by selective substitution. In this work, the selective substitution of In+ by Pb for the binary InTe material monotonically reduces the carrier concentration, which is greatly beneficial to the mechanism investigation of serious grain boundary scattering (GBS). This is the first time this point has been mentioned with regard to InTe material. As a result, we found that GBS was dominated by the grain size when the carrier concentration was higher than 0.7 × 1019 cm−3 but was inversely governed by the carrier concentration when the carrier was situated at a lower density. In particular, the occupation of Pb on the targeted In+ site could further reduce the lattice thermal conductivity. Finally, In0.9999Pb0.0001Te achieved the improved power factor and average zT value, which could contribute to high-power generation below a medium temperature. This effect of increasing the carrier concentration on the suppression of GBS sheds light on the possibility of improving electron mobility by increasing the carrier concentration.