Tailoring the formation and distribution of loose-packing regions and dense-packing regions in the bulk metallic glasses (BMGs) has long been considered to facilitate macroscopic mechanical properties and break strength-ductility trade-off. Herein, studies on structural heterogeneity caused by the deep cryogenic treatment (DCT) were systematically carried out to not only probe the atomic-scale structure-property relationships but also further understand the deformation mechanisms involving improved plasticity without sacrificing strength on the basis of these experimental results in rejuvenated Zr-based BMGs. The internal stress due to the cooling contraction will activate the atomic motion in weakly bonded regions and meanwhile form perfect icosahedral short-range ordering (ISRO) via containing mobile atoms to become denser geometric configurations. The multiplication and propagation of shear bands are closely associated with their structural features originating from the optimization of perfect icosahedral ordered structures in the rejuvenated glassy matrix. It is believed that these findings are helpful to shed light on the correlations between atomic-scale structural heterogeneity and macroscopic mechanical properties and further provide new possibilities for the development of promising BMGs with strength-ductility synergy.