In arid and semiarid areas, the effects of afforestation on soil organic carbon (SOC) have received considerable attention. In these areas, in fact, soil inorganic carbon (SIC), rather than SOC, is the dominant form of carbon, with a reservoir approximately 2–10 times larger than that of SOC. A subtle fluctuation of SIC pool can strongly alter the regional carbon budget. However, few studies have focused on the variations in SIC, or have used stable soil carbon isotopes to analyze the reason for SIC variations following afforestation in degraded semiarid lands. In the Mu Us Desert, northwest China, we selected a shifting sand land (SL) and three nearby forestlands (Populus alba) with ages of 8 (P-8), 20 (P-20) and 30 (P-30) years, and measured SIC, SOC, soil organic and inorganic δ13C values (δ13C-SOC and δ13C-SIC) and other soil properties. The results showed that SIC stock at 0–100 cm in SL was 34.2 Mg ha-1, and it increased significantly to 42.5, 49.2, and 68.3 Mg ha-1 in P-8, P-20, and P-30 lands, respectively. Both δ13C-SIC and δ13C-SOC within the 0–100 cm soil layer in the three forestlands were more negative than those in SL, and gradually decreased with plantation age. Afforestation elevated soil fine particles only at a depth of 0–40 cm. The entire dataset (260 soil samples) exhibited a negative correlation between δ13C-SIC and SIC content (R2 = 0.71, P < 0.01), whereas it showed positive correlation between SOC content and SIC content (R2 = 0.52, P < 0.01) and between δ13C-SOC and δ13C-SIC (R2 = 0.63, P < 0.01). However, no correlation was observed between SIC content and soil fine particles. The results indicated that afforestation on shifting SL has a high potential to sequester SIC in degraded semiarid regions. The contribution of soil fine particle deposition by canopy to SIC sequestration is limited. The SIC sequestration following afforestation is very probably caused by pedogenic carbonate formation, which is closely related to SOC accumulation. Our findings suggest that SIC plays an important role in the carbon cycle in semiarid areas and that overlooking this carbon pool may substantially lead to underestimating carbon sequestration capacity following vegetation rehabilitation.