The high temperature brittle zone of cryogenic high manganese (Mn) steel often leads to transverse cracks during hot working process, which seriously hinders the production of its sheets or wires. In the present work, we found that the brittle zone (at temperatures between 800 and 900 °C) can be effectively enhanced by homogenizing the initial microstructure and composition. It was revealed that the elements such as Mn and C tend to be enriched in the inter-dendrite area of the as-cast samples, which is difficult to eliminate even during high temperature forging. The C segregation promotes the formation of fine precipitates, such as NbC and Cr23C6. These carbides are usually located at or near grain boundaries, which can weaken grain boundary cohesion and deteriorate the hot ductility. However, homogenizing the initial composition helps to increase the size of hybrid precipitates containing TiN, WC, MoO2 and WO3 and inhibit the precipitation at grain boundaries. Meanwhile, homogenizing the initial microstructure size and stored energy reduces the critical dynamic recrystallization (DRX) temperature to 850 °C. A lower Z value and critical DRX stress increase the DRX fraction from 24% to 66%, which recover microstructure and inhibit grain boundary slip.