Abandoned cropland and impervious surface expansion are generally considered to have positive and negative impacts on net ecosystem productivity (NEP), respectively. However, the comprehensive comparison of the impacts of these two land use changes on NEP remains lacking. We use multi-year land cover data, NEP data, and statistical data to track the interannual variability of abandoned cropland and impervious surface expansion in China, and subsequently estimate their impacts on NEP. From 2000 to 2019, 30.8 × 106 ha of cropland in China was abandoned, while the impervious surface area expanded by 8.97 × 106 ha over the period. The average annual abandonment rate and impervious surface expansion rate were 0.83% and 2.96%, respectively. The average annual NEP of abandoned cropland and impervious surface expansion were 39.9 g C/m2 and 16.2 g C/m2, respectively. Using the space-for-time method to estimate NEP changes, results showed that the carbon sink of 0.013 Tg C accumulated by abandoned cropland not only offset the carbon source of 0.01 Tg C from impervious surface expansion but sequestered additional 0.003 Tg C during the study period. Further analysis of the relationships between NEP and natural and anthropogenic factors revealed that temperature had the most significant positive impact on NEP (0.35, p < 0.01), while the urbanization rate had the most significant negative impact on NEP (−0.33, p < 0.01). This study provides a new insight for regulating the effects of land use changes on natural carbon sink/source, and highlights the importance of abandoned cropland to enhancing natural carbon sink.