Vegetation restoration not only extensively reshapes spatial land use patterns but also profoundly affects the dynamics of runoff and sediment loss. However, the influence of vegetation restoration on runoff and sediment yield from a regional perspective are scarce. This study therefore focused on 85 sites within the “Grain for Green” Project (GGP) region on the Loess Plateau, to investigate the impacts of the GGP on soil erosion. The results revealed a notable reduction in sediment loss and runoff due to vegetation restoration. Since the inception of the GGP in 1999, approximately 4.1 × 106 ha of degraded lands have been converted into forestlands, shrublands, and grasslands, resulting in an average annual reduction of 1.4 × 109 m3 in runoff and a decrease of 3.6 × 108 t in annual sediment loss on the whole Loess Plateau, with the GGP contributing approximately 26.7% of the sediment reduction in the Yellow River basin. The reduced soil erosion has mainly been regulated by vegetation cover, soil properties (clay, silt, and sand), slope, and precipitation on the Loess Plateau. The insights gained offer valuable contributions to large-scale assessments of changes in soil erosion in response to vegetation reconstruction and enhance our understanding of the spatial configurations associated with soil erosion control measures.