The Southwest Alpine Canyon Area (SACA), a well-known ecological vulnerability region, plays a very important role in China. Identifying the driving force of the spatial heterogeneity of vegetation and the response of interannual vegetation changes to climate change and human activities would be helpful for ecosystem management. Based on the NDVI dataset, the study analyzed the trend of NDVI change from 2000 to 2019 using the Theil–Sen trend analysis and the Mann–Kendal significance test, detected the driving forces of the spatial heterogeneity of NDVI by the means of the geographical detector, and analyzed the relative contribution of climate change and human activities to interannual NDVI changes using residual analysis model. The results showed that, in terms of the spatial distribution, the pattern of NDVI showed that it is higher in the southeast and lower in the northwest region of the SACA. Elevation was the dominant factor influencing the spatial heterogeneity of NDVI, with the explanatory power of 64%, much larger than other factors, and vegetation type, temperature, precipitation, land use type, and soil type were the main factors. In addition, the explanatory power of the dual factor interaction was higher than that of the single factor effect, which showed two kinds of interaction relationships: bivariate enhancement and nonlinear enhancement. In terms of the temporal variation, 85.59% of the study area showed an increasing trend, and only 14.41% of the area showed a decreasing trend. The main factor affecting NDVI changes was human activities, and climate change was the secondary factor, with relative contributions of 71.35% and 28.65%, respectively. The study will promote a better understanding of the complex mechanisms of vegetation changes and provide scientific recommendations for the prevention of vegetation degradation and vegetation restoration in the SACA.