Soil erosion is a global environmental problem, and soil conservation is the prevention of soil loss from erosion. The Ten Kongduis (kongdui is the translation of “short-term flood gullies” in Mongolian) are ten tributaries in the upper Inner Mongolia section of the Yellow River Basin. The study of the spatial and temporal variability in soil conservation in the Ten Kongduis is of extraordinary scientific significance both in terms of the discipline and for the ecological and environmental management of the region. With the InVEST model, the characteristics of the spatial and temporal variations in soil conservation service in the Ten Kongduis since 2000 and how rainfall and land use have influenced soil conservation were analyzed. The results show that both avoided erosion and avoided export varied considerably between years. The minimum values of avoided erosion and avoided export were both in 2015, with values of 17.59 × 106 t and 0.92 × 106 t, respectively. The maximum value of avoided erosion was 57.03 × 106 t in 2020 and that of avoided export was 4.08 × 106 t in 2000. Spatially, avoided export was primarily found in the upper reaches of the east–central portion of the study area, and avoided erosion, with values of >40 t·(ha·yr)−1, was in the upper east–central portion of the study area, followed by the upper west–central portion. The difference between upstream and downstream was larger in the western part of the study area. The effect of rainfall was dominant and positive in both avoided erosion and avoided export. The relationships between the rain erosivity factor and the values of avoided erosion and avoided export were significantly positive. Where more erosion occurs, more erosion is retained. Soil that has been eroded away from slopes under vegetation or other water conservation measures may not necessarily be transported to the stream channel in the current year. These conclusions will help us to have a clearer understanding of where sediments are generated and transported and provide a scientific basis for soil and water conservation and ecosystem safety management of watersheds.