TeO2 single crystals have been widely used in military and civilian fields as an excellent acousto-optic (AO) material. However, due to its significant thermal expansion anisotropy and poor mechanical properties, there is a considerable risk of cracking, which hinders further applications in the AO field. Here we successfully grew a TeO2 single crystal doped with 2 % Mn4+ ions (TeO2:0.02Mn4+). This single crystal demonstrates excellent characteristics, including lower acoustic attenuation (α) and larger refractive index values. In addition, it also possesses a higher AO figure of merit (M2) of 832 × 10−15 s3/kg than TeO2 single crystal (∼793 × 10−15 s3/kg). More importantly, the TeO2:0.02Mn4+ single crystal demonstrates low thermal expansion anisotropy within a temperature range of up to 450 °C, indicating a greater practical value. The high AO performance may be attributed to manganese ion doping. The doping of Mn increased the refractive indices by enhancing the electronic polarizability and achieved lower acoustic attenuation by improving mechanical properties. Combining the results of elastic modulus and Raman spectroscopy analysis, it is observed that manganese doping optimizes crystal structure disorder, thus improving the issue of thermal expansion anisotropy. These results indicate that this TeO2:0.02Mn4+ single crystal has significant potential in designing and manufacturing AO devices.