Initiation and evolution of flow units and their correlation with mechanical behaviors of metallic glasses (MGs) are well known significant for understanding the rheological behavior of glassy substances. Here, the evolution of plastic deformation behavior of a typical Zr-based MG induced by ultrasonic vibration is investigated by means of nanoindentation, especially focusing on studying structural evolution through analysis of load-displacement curves. Furthermore, the micro-morphologies of residual indentations are characterized by using the scanning electron microscope (SEM) and atomic force microscope (AFM). Compared to quasi-static nanoindentation tests, a significant reduction in the hardness and elastic modulus of the MG by about 15 % and 30 %, respectively, is observed when subjected to ultrasonic vibration nanoindentation with an amplitude of 0.4 μm, accompanied by a pronounced indentation size effect. In addition, higher ultrasonic vibration amplitude induces larger indentation depth, more pronounced serrated flows in the load-displacement curves and more remarkable shear bands around the residual indentations, indicating ultrasonic vibration-promoted plastic flows in MG. This study provides an insight into the structural origin of plastic flows in MG systems under ultrasonic vibration, which will help to facilitate their application in the fields such as ultrasonic vibration assisted forming.