Terrestrial gross primary production (GPP) changes due to impervious surfaces significantly impact ecosystem services in watersheds. Understanding the asymmetric response of vegetation GPP to impervious surface expansion is essential for regional development planning and ecosystem management. However, the asymmetric response of vegetation GPP to the impacts of impervious surface expansion is unknown in different watersheds. This paper selected the Yellow River and Yangtze River basins as case studies. We characterized the overall change in GPP based on changes in impervious surface ratio (ISR), determined impervious surface expansion's direct and indirect impacts on GPP in the two watersheds, and further analyzed the asymmetric response of the compensatory effects of indirect influences on the impervious surface expansion in different watersheds. The results showed that: (1) The vegetation GPP decreased with increasing ISR in the Yangtze River Basin, while that in the Yellow River Basin first increased and then reduced. (2) The direct impacts of increased ISR reduced vegetation GPP, while the indirect impacts both had a growth-compensating effect. Growth compensation stabilized at approximately 0.40 and 0.30 in the Yellow and Yangtze River Basins. (3) When the ISR was 0.34–0.56, the growth compensation could offset the reduction of GPP due to direct impact and ensure that the background vegetation GPP was not damaged in the Yellow River Basin. In contrast, the background vegetation GPP was inevitably impaired with increased ISR in the Yangtze River Basin. Therefore, this study suggests that the ISR should be ensured to be between 0.34 and 0.56 to maximize the impervious surface of the Yellow River Basin without compromising the background vegetation GPP. While pursuing impervious surface expansion in the Yangtze River Basin, other programs should be sought to compensate for the loss to GPP.