Converting natural vegetation for agriculture has resulted in the loss of approximately 5% of the current global terrestrial soil organic carbon (SOC) stock to the atmosphere. Increasing the agricultural area under grassland may reverse some of these losses, but the effectiveness of such a strategy is limited by how quickly SOC recovers after conversion from cropland. Using soil data and extensive land-use histories gathered during the national German agricultural soil inventory, this study aims to answer three questions regarding agricultural land-use change (LUC): (i) how do SOC stocks change with depth following LUC; (ii) how long does it take to reach SOC equilibrium after LUC; and (iii) what is the legacy effect of historic LUC on present day SOC dynamics? By using a novel approach that substitutes space for time and accounts for differences in site properties using propensity score balancing, we determined that sites that were converted from cropland to grassland reached a SOC equilibrium level 47.3% (95% confidence interval (CI): 43.4% to 49.5%) above permanent cropland levels 83 years (95% CI: 79 to 90 years) after conversion. Meanwhile, sites converted from grassland to cropland reached a SOC equilibrium level -33.6% (95% CI: -34.1% to -33.5%) below permanent grassland levels after 180 years (95% CI: 151 to 223 years). We estimate that, over the past century, today's German agricultural soils (16.6 million ha) have gained about 40 million Mg C. Furthermore, croplands with historic LUC from grassland are losing SOC by -0.26 Mg ha-1 year-1 (10% of agricultural land) while grasslands historically converted from cropland are gaining SOC by 0.27 Mg ha-1 year-1 (18% of agricultural land). This study shows that even long-standing temperate agricultural sites likely have ongoing SOC change as a result of historical LUC.