TIMETAL 54M is a newly developed (α + β) titanium alloy with nominal composition Ti-5Al-4V-0.6Mo-0.4Fe. The alloy can provide a cost benefit over Ti-6Al-4V due to improved machinability and formability. In the present work, evolution of mechanical properties in terms of tensile and hardness values is investigated as a function of deformation degrees imposed via rotary swaging (RS). Microstructure, mechanical properties, and fatigue performance of Ti-54M are investigated after severe plastic deformation by RS conducted at 850 °C and after being subjected to two different post-swaging annealing conditions. Optical microscopy and scanning electron microscopy using electron back scatter diffraction were utilized to document the evolution of the microstructure. Tensile tests were conducted to characterize mechanical properties. RS, to a true strain of 3.0, is found to lead to a marked ultrafine-grained structure of about 1 μm grain size with low content of high angle grain boundaries (HAGBs). Post-swaging heat treatment at 800 °C followed by air cooling did not change the grain size but exhibited high content of HAGBs. Post-swaging heat treatment at 940 °C followed by furnace cooling resulted in a grain size of about 5 μm and enhanced work-hardening capability and ductility, which resulted in less fatigue notch sensitivity, but at the same time lower fatigue strength at 107 cycles.