Grain refinement is an important technique to improve the mechanical properties of metal materials. In this study, a novel technique (in-situ oxidation process) was proposed. It successfully refined grain and improved the mechanical properties of AZ31 alloy. The effect of different melt treatment process on AZ31 alloy was comparatively studied. For the in-situ oxidation process, effects of flow of O2/Ar mixed gas and the inoculating temperature on grain size were investigated. The results showed that in-situ oxidation process caused an available grain refinement of AZ31 alloy, whereas the only oxidation or the only stirring processes was invalid for grain refinement. After being treated by in-situ oxidation process with an O2/Ar mixed gas flow of 10/100 ml/min, average grain size of AZ31 alloy decreased from 1146 to 554 μm. There existed an effective grain refinement only when O2 gas flow ranges from 5 to 10 ml/min. Oxides (MgO and MgAl2O4) were found in the in-situ oxidation process refined AZ31 alloy. E2EM calculation results indicated that MgO and MgAl2O4 possess a high nucleating potency for α-Mg grain. Measured cooling curves showed that the initial nucleation undercooling declined and the nucleation process ended early. Hence, grain refinement of AZ31 alloy by in-situ oxidation process was attributed to the heterogeneous nucleation of α-Mg grain on MgO and MgAl2O4.