Grassland ecosystem dominates in the Mongolian Plateau, mostly located in its arid and semi-arid regions. Although the ecosystem is an important source for agriculture, it is also a fragile system ecologically. This system is one of the most sensitive areas to global climate change. Precipitation (PPT) and soil moisture (SM) are important water sources in the grassland ecosystem, and their changes would greatly affect vegetation growth. This paper generates the precipitation use efficiency (PUE) and soil water use efficiency (SWUE) of Mongolian Plateau grassland based on multi-source remote sensing data to investigate the spatio-temporal distribution pattern and identify the driving factors. Results showed four main findings. Firstly, two water use efficiency (WUE) indicators show a generally increasing trend from 2000 to 2018, with average PUE and SWUE 1.07 gC·m−2·mm−1 and 1.03 gC/kg·H2O, respectively. They have similar spatial distribution patterns, consistent with the available water resources, decreasing from northeast to southwest. However, grassland vegetation growth is more sensitive to soil moisture than precipitation, and the dynamic change of SWUE is smoother and more significant than PUE. Secondly, due to the higher species richness, better vegetation biological characteristics and less severe growth environment, meadow grassland has the highest PUE and SWUE, followed by typical grassland and desert steppe. Thirdly, PUE and SWUE are relatively low in extremely arid and humid regions. In areas with relatively moderate water conditions (PPT in 148–360 mm, SM in 0.14–0.35 cm3/cm3), two indicators increase with the abundance of moisture conditions and reach the maximum. Fourthly, there is a positive linear relationship between PUE (SWUE) with precipitation and a unimodal correlation between PUE (SWUE) with temperature across the entire grassland. However, a varying correlation exists in different grassland ecosystems, especially meadow grassland. Together with analyzing past and future trends, this study provides strong evidence to reflect the impact of global climate change and the management and protection of the grassland ecosystems in arid and semi-arid regions.