The Mg–Sc alloys have attracted considerable attention from researchers due to their martensitic transformation behavior. However, the relatively low yield strength limits their practical application. Thus, Zn alloying combined with annealing treatment was used to improve the mechanical properties by modulating its microstructure. Zn alloying can effectively delay the recrystallization rate and refine the grain size of the alloy. The β/α intensity ratio and grain size of β phase increase with the increasing annealing temperature. When the annealing temperature is 600 °C, the major phase is β-Mg-Sc with minor α-Mg-Sc, ScZn and Mg3Sc phases. Further increasing the temperature to 650 °C would lead to the dissolution of ScZn and Mg3Sc phases into the matrix, resulting in smaller precipitations. The Mg-Sc-Zn alloy annealed at 600 °C for 60 min owns the best comprehensive mechanical properties, with yield strength, ultimate tensile strength, and elongation of 307.2 MPa, 330.5 MPa, and 21.1%, respectively. Further TEM analysis reveals that fine grain reinforcement, precipitation reinforcement, and stress-induced martensitic transformation during tensile deformation are the main factors for the enhancement of strength and ductility in this alloy.