A new method of machining ultrasonic cavitation abrasive flow based on phase control technology was proposed for improving the machining efficiency of the inner wall of TC4 (Ti-6Al-4V) titanium alloy tubes. According to ultrasonic phase control theory and Hertzian contact theory, a model of ultrasonic abrasive material removal rate under phase control technology was established. Using COMSOL Multiphysics 6.1 software, the phase control deflection effect, acoustic field distribution, and the size of the phase control cavitation domain on the inner wall surface were examined at different transducer frequencies and transducer spacings. The results show that the inner wall polishing has the most excellent effect at a transducer frequency of 21 kHz and spacing of 100 mm. In addition, the phased deflection limit was explored under the optimal parameters, and predictive analyses were performed for voltage control under uniform inner wall polishing. Finally, the effect of processing time on polishing was experimented with, and the results showed that the polishing efficiency was highest from 0 to 30 min and stabilized after 60 min. In addition, the change in surface roughness and material removal of the workpiece were analyzed under the control of the voltage applied, and the experimental results corresponded to the voltage prediction analysis results of the simulation, which proved the viability of phase control abrasive flow polishing for the uniformity of material removal of the inner wall of the tube.