High strength titanium alloys suffer from a limited combination of strength and ductility. Considering their microstructure, the coarse β grain is a detrimental factor. In the present study, the effects of hot deformation and annealing treatment on the β grain refinement of Ti–5Al–5Mo–5V–1Cr–1Fe (TC18) titanium alloy have been investigated by using electronic backscattered diffraction (EBSD) and transmission electric microscopy (TEM). Hot compression tests of TC18 alloy were performed between 1073 and 1223 K at strain rates of 0.01 and 0.10 s−1. Subsequent annealing treatments were conducted to investigate the effect of deformation microstructure on the recrystallization behavior of β grains. The results indicate that dynamic recovery (DRV) is the dominant restoration mechanism during hot deformation. Discontinuous dynamic recrystallization (dDRX) is significantly limited at a higher temperature in the β region, whereas continuous dynamic recrystallization (cDRX) of the β phase is observed at a higher strain rate in the (α + β) region. The final annealing microstructure is highly dependent on the deformation microstructure. Limited dDRX during hot deformation could not effectively refine the β grains. The refined and uniform β grains with an average grain size of 12 μm are achieved by combining hot deformation and subsequent annealing in the (α + β) region. This study provides a guide for the β grain refinement of high strength titanium alloys.