Carbonatitic magmatism plays a significant role in Earth's carbon cycle, which is also a lithoprobe of crust-mantle interaction, mantle metasomatism and partial melting. Due to different mineral assemblages and geochemical compositions, and diverse tectonic settings, the origin of carbonatite has long been debated. At subduction zones, sediments (including carbonates) are subducted into the mantle with the downgoing oceanic slab. However, the detailed mechanism of how subducted carbonates contribute to carbonatitic magmatism remains unclear. Here we present geochronological, geochemical and isotopic study on the Taohuala Mountain carbonatite at the southern margin of the Alxa Block, North China Craton. The classification of carbonatite from the Taohuala Mountain relies strongly on the observations of obvious intrusion contact relationships and flow structures in field outcrop. The Taohuala Mountain carbonatite has SiO2 ranging from 2.37wt.% to 11.45wt%, high CaO (45.93–53.86wt%) and low MgO (0.51–4.39wt%), and is characterized by enrichment of LILE (Ba, Sr), depletion of HFSE (Nb, Ta, Zr, Hf), and slightly negative Ce and Eu anomalies. Carbonates in the samples have high 87Sr/86Sr (0.70686–0.70694) and low 143Nd/144Nd (0.511635–0.511924). Remarkably, the highly fractionated δ18OVSMOW (11.83–25.92‰) indicates components of both sedimentary and mantle origin. Detailed zircon in situ U-Pb dating and oxygen isotope analysis exhibit contrast ages and δ18OVSMOW from core to rim, i.e., old ages (mainly>800Ma), high Th/U (mainly>0.5) and low δ18OVSMOW (6.37–11.44‰) in cores (inherited), whereas young ages (~400Ma), low Th/U (mainly<0.01) and high δ18OVSMOW (20.04–24.54‰) in rims, suggesting that the Taohuala Mountain carbonatite may have been generated from melting of subducted sedimentary carbonates. Considering all these evidences, and that the collision along Qilian Mountains was older than the carbonatite, we propose that a large volume of sedimentary carbonates subducted and remained in the lithospheric mantle under the Alxa block during the closure of the Paleo-Qilian Ocean. Subsequently, the carbonatite was formed by melting of carbonates with minor contributions from the mantle during the breakoff or rollback of the Paleo-Asian oceanic slab.