O3 pollution in China has been increasing in recent years, but the process of O3 impact on net primary productivity of terrestrial ecosystems remains unclear. We attempts to explore a remote sensing-based method to assess the impact of O3 on NPP of China’s terrestrial ecosystems by combining MODIS NPP and the latest ground observation data of O3 concentration. By comparing the NPP data of MODIS image pixels with the 6-year average AOT40 data of corresponding pixels, we extracted the signal data that highlighted the effects of O3 on NPP and established the response relationships between AOT40 and NPP. It was found that NPP was significantly negatively correlated with AOT40 in farmland and grassland ecosystems in China (farmland: r = 0.8674, p < 0.003, grassland: r = 0.7181, p < 0.03). Then the response relationships were adopted to evaluate the effect in China in 2014. The results showed that the estimated percentage of O3-induced NPP decline was in the range of 5%–35%. Among them, the most significant declines were found in farmland ecosystems, with the vast majority of declines in 10%–35%. The decrease rate of evergreen coniferous forests ranked second, mostly in the range of 15%–20%. The grassland ecosystems declined at a lower rate, almost between 5% and 15%. And the evergreen broad-leaved forests has the lowest decline rate, most distributed in 0%–5%. The highest percentage decreases were mainly found in the Beijing-Tianjin-Hebei region and Shandong Province. And the decline rate of farmland ecosystems was significantly higher than other ecosystem types.