In this study, the stress–strain responses of G115 martensitic steel, Inconel alloy 750H, and 316H austenitic steel were thoroughly investigated at elevated temperatures via low cycle fatigue tests. The evolutions of effective stress and back stress were determined by stress partition method and were related to cumulative plastic strain. The mobile dislocation density, dislocation structure, and sub-grain structure were discussed to reveal the effect of microstructure on stress–strain responses during fatigue process. Furthermore, a unified elastic–plastic framework was established by introducing a peak plastic strain-dependent relaxation factor, a cumulative plastic strain-modified kinematic hardening model, and a modified isotropic hardening model. The validity of the unified model was discussed based on the maximum stress evolution, stress rate factor, and hysteresis loop. Good consistencies were observed between the experimental and predicted results and showed its strong capability to integrate continuous softening, hardening–softening, and hardening–softening-secondary hardening behaviors into a constitutive model.