One of the practices often mentioned to achieve climate change mitigation is the long-term cultivation of perennial plants. The objective of the study was to estimate changes in the accumulation of soil organic carbon (SOC) and its fractions in 0–10, 10–20, 20–30 cm, and within 0–30 cm soil layer of red fescue (Festuca rubra L.) swards that differ in age (5, 10 and 15 years) as well as to compare them with the arable field. Our results show that SOC accumulation at 5-year-old cultivation of red fescue is high, later this SOC increase slowed down from 71% in the 0–30 cm soil layer when land use was converted from arable field to 5-year-old sward to 1% from 10 to 15 years. The level of water extractable organic carbon (WEOC) in the 0–30 cm soil layer of swards was significantly higher compared to the arable field. The positive effect of these swards in the accumulation and stabilization of organic carbon during humification in the soil was also determined. The largest amounts of mobile humic substances (MHS) and mobile humic acids (MHA) accumulated in the 0–10 cm layer of sward soil (3.30–4.93 and 1.53–2.48 g kg−1, respectively). In conclusion, the findings suggest that a conversion from arable to soil under permanent grass cover significantly improves carbon status.