The properties of a material can be engineered by manipulating its atomic and chemical architecture. Nanoglasses which have been recently invented and comprise nanosized glassy particles separated by amorphous interfaces, have shown promising properties. A potential way to exploit the structural benefits of nanoglasses and of nanocrystalline materials is to optimize the composition to obtain crystals forming within the glassy particles. Here, a metastable Fe-10 at% Sc nanoglass is synthesized. A complex hierarchical microstructure is evidenced experimentally at the atomic scale. This bulk material comprises grains of a Fe90 Sc10 amorphous matrix separated by an amorphous interfacial network enriched and likely stabilized by hydrogen, and property-enhancing pure-Fe nanocrystals self-assembled within the matrix. This composite structure leads a yield strength above 2.5GPa with an exceptional quasi-homogeneous plastic flow of more than 60% in compression. This work opens new pathways to design materials with even superior properties.