Grain size stabilization is crucial for the production and application of nanocrystalline (NC) materials. The mechanically alloyed (MA) NC Fe-Zr system is known as a very successful NC system as it exhibits excellent thermal stability at elevated temperatures. The grain size stabilization of this system has been previously ascribed to its reduced grain boundary (GB) energy by Zr segregation and Zener pinning of Zr-rich intermetallic precipitates. In this work, we report a different mechanism that significantly contributes to grain size stabilization of this NC alloy system using two MA-produced NC Fe-Zr alloys (Fe-1 at.% Zr and Fe-5 at.% Zr) as examples. We show by using atom probe tomography and Cs-corrected transmission electron microscopy that highly dispersed coherent Fe-Zr-O nanoclusters, with a number density up to 1024 m−3, form in ferrite matrix after annealing at certain temperatures. Our first-principles calculations indicate that the formation of these nanoclusters is caused by the ordering of Zr and O-impurity in ferrite matrix. We analyzed the underlying mechanism of grain size stabilization in terms of the experimental results and the Zener pinning theory, and suggest that the pinning effect exerted by these nanoclusters significantly contributes to grain size stabilization of the NC Fe-Zr alloys.