Direct epitaxial growth of III-V materials on complementary metal-oxide-semiconductor (CMOS)-compatible Si substrates has long been a scientific and engineering problem for next-generation light-emitters and non-volatile memories etc. The challenges arise from the lattice mismatch, thermal mismatch, and polarity mismatch between these materials. We report a detailed study of growing high-quality GaSb epilayers with low defect density on on-axis silicon substrates by interface engineering through all-molecular beam epitaxy (MBE) technology. We also systematically investigated the defect self-annihilation mechanism of GaSb epitaxially grown on on-axis Si (001) substrates. It was found that the misfit dislocation array was formed at the interface of AlSb/Si; threading dislocations and antiphase domain boundary annihilated at the initial GaSb layer promoted by the high-density AlSb islands, which was confirmed by transmission electron microscopy (TEM) results. Finally, a 2 µm GaSb epilayer with a step-flow surface, root-mean-square (RMS) roughness of 0.69 nm, and a rocking curve full width at half maximum (FWHM) of 251 arcsec was obtained. The photoluminescence in the near-infrared region of the GaSb/AlGaSb quantum well grown on Si substrate was also demonstrated. Our results highlighted the possible step towards the all-MBE direct growth of Sb-based infrared optoelectronic and microelectronic devices on CMOS-compatible Si substrates.