Exploring the structural and physical properties of new vanadium dioxide (VO2) allotropes has attracted considerable interest because of the structure diversity and unique physical properties of VO2. Here, we demonstrate a reversible pressure-induced structural transition and metallization of the novel metastable polymorph VO2(Mx′) and a thermally driven structural transition from VO2(Mx′) to the monoclinic phase VO2(M1) at relative low temperature based on X-ray diffraction (XRD) and Raman and infrared spectroscopy. It is shown that the metastable phase VO2(Mx′) undergoes the structural transitions of VO2(Mx′)–(12 GPa) VO2(Mx′′)–(30–80 GPa) VO2(X) upon compression, obviously different from the pressure-induced amorphization observed in other metastable phases VO2(A) and VO2(B). Moreover, the IR data demonstrated that the pressure-induced metallization (PIM) occurs in the VO2(Mx′′) phase at about 40 GPa, which is mainly associated with electron–electron correlations. Further analysis suggests that all of the sample transforming into the same high-pressure VO2(X) phase with the stable M1 phase could mainly result from the VO6 octahedra and empty spaces between VO6 octahedra in their intermediate high pressure phases VO2(Mx′′) and VO2(M1′) following similar variations under pressure. These findings present new insight into the differences of structural transitions and physical properties between the stable and metastable phases of transition-metal oxides under pressure.