To cope with climate change, it is of great importance to describe the temporal and spatial evolution of climate change vulnerability and its driving factors. Therefore, this paper establishes a comprehensive index of vulnerability to climate change based on the vulnerability scoring diagram (VSD) framework. Moran's I index is used to study the spatial evolution characteristics of vulnerability, and spatial regression analysis is used to explore the factors influencing the spatial distribution of vulnerability. The results show that (1) the climate change vulnerability of China has decreased over time, and the sensitivity state is relatively stable; however, the annual change in exposure and adaptive capacity is significant. (2) The western region of China is more vulnerable than the eastern region, and the most vulnerable provinces are Guizhou and Gansu. (3) The regional vulnerability is generally in a significant spatial agglomeration state. (4) Finally, the driving factors of the spatial distribution of climate change vulnerability include forest coverage, the urban-rural income gap and information technology. These recommendations provide detailed discussions and scientific information for mitigating global warming and formulating long-term emission reduction targets, thereby optimizing resource allocation and providing spatial governance directions for the formulation of adaptation policies.