The evaluation of water conservation function of the ecosystem and the analysis of influencing factors are conducive to the implementation of "Grain for Green" Projects, scientific management of land use types, and the optimization of ecosystem service functions. In this paper, we selected the data of precipitation, soil, evapotranspiration, land use/cover change, and other data to quantitatively evaluate the water conservation function of the ecosystem in Shaanxi Province from 1980 to 2015 and explored the key factors which lead to the spatial differences in water conservation functions. The results showed that (1) from 1980 to 2015, the water production and water conservation of Shaanxi Province's ecosystem based on grid units decreased by 12.81 × 108 m3 and 7.74 × 108 m3, respectively. (2) The use types of cultivated land, forest land, and grassland had a prominent contribution to the water production and water conservation of the ecosystem, while the contribution rate of water area, construction land, and unused land was small. During the research period, "Grain for Green" Projects consumed a lot of water production in a short period of time, which was not conducive to the improvement of water production capacity in Shaanxi Province. However, the conversion of cultivated land to grassland would help improve the water conservation capacity of the ecosystem, increasing the water conservation capacity by 0.77 × 108 m3. (3) The water conservation function of Shaanxi Province was "hot in the south and cold in the north," with obvious regional and spatial heterogeneity from 1980 to 2015. (4) The factors of precipitation, evapotranspiration, slope, common cultivated land area, afforestation area, and water conservation function showed a positive correlation, and the impact was significant, while the overall vegetation coverage index showed a negative correlation with it. Therefore, it is necessary to comprehensively consider the local climatic and environmental conditions to implement ecological protection and restoration policies in terms of enhancing water conservation function of the ecosystem. (5) Targeted measures are proposed for different areas in Shaanxi Province: The Northern Shaanxi area should be regarded as an ecological restoration and management area, focusing on returning farmland to grassland; Central Shaanxi Plain should be considered as an ecological protection zone, and the key task is to effectively protect the ecological land; the Southern Shaanxi area should be regarded as a key water conservation area, and the main function of the forest and grass ecosystem to conserve water is to be exerted.