GeMnTe2 is considered to be an excellent thermoelectric material because it is cheaper and easier to obtain than GeTe, has a higher carrier concentration than MnTe, and its cubic structure is not prone to phase transition and is relatively stable. The aim of this paper is to optimize the carrier concentration of GeMnTe2 and reduce the thermal conductivity by Charge Balance Vacancy Engineering, thus giving the material higher thermoelectric properties. In this paper, GeMnTe2 has obtained better thermoelectric properties by Charge Balanced Vacancy Engineering. Sb is introduced into GeMnTe2 as an effective dopant, which leads to an increase in the density of states near the Fermi level. In the heat, when Sb doping concentration was 0.1 and the actual content of Ge was 0.85, the highest power factor was 13.72 μWcm−1K−2, which was 17.70% and was head and shoulders above the pure. The ZT value was 0.67 and the maximum value of Ge0.97MnSb0.02Te2 sample was 0.8 at the pure phase condition of 770 K, which was about 19.94% higher than that of GeMnTe2, which proved that Charge Balance Vacancy Engineering is an effective technique to improve the thermoelectric performance of GeMnTe2 thermoelectric devices.