The dynamics of terrestrial vegetation have changed a lot due to climate change and direct human interference. Monitoring these changes and understanding the mechanisms driving them are important for better understanding and projecting the Earth system. Here, we assessed the dynamics of vegetation in a semi-arid region of Northwest China for the years from 2000 to 2019 through satellite remote sensing using Vegetation Index (VI) data from the Moderate Resolution Imaging Spectroradiometer (MODIS), and analyzed the interannual covariation between vegetation and three climatic factors—air temperature, precipitation, and vapor pressure deficit (VPD)—at nine meteorological stations. The main findings of this research are: (1) herbaceous land greened up much more than forests (2.85%/year vs. 1.26%/year) in this semi-arid region; (2) the magnitudes of green-up for croplands and grasslands were very similar, suggesting that agricultural practices, such as fertilization and irrigation, might have contributed little to vegetation green-up in this semi-arid region; and (3) the interannual dynamics of vegetation at high altitudes in this region correlate little with temperature, precipitation, or VPD, suggesting that factors other than temperature and moisture control the interannual vegetation dynamics there.