The Central Asian Orogenic Belt was formed by the subduction to closure of the Paleo-Asian Ocean (PAO). However, it is highly controversial about the closing time of the PAO, especially its middle segment in the northern Alxa orogenic belt (NAOB). In this study, the new and published zircon U–Pb and Hf data for the Carboniferous to Permian sediments across the NAOB have been integrated to reply the above problem. The depositional ages have been constrained as the Carboniferous to Permian by the detrital zircon ages, fossil assemblages and stratigraphic correlation. The Carboniferous sandstones are dominated by the Paleozoic zircons (mainly around 380–510 Ma) with a few Precambrian zircons. The late Cambrian to early Carboniferous zircons with positive to slightly negative εHf(t) values were probably sourced from the orogen itself. The early Paleozoic zircons with slightly to extremely negative εHf(t) values and the late Archean to Paleoproterozoic zircons were likely derived from the surrounding cratonic blocks in the south. For the Permian samples, the Carboniferous to Permian age signal is enhanced. The Permian zircons yield similar age peaks around 278–279 Ma and similar εHf(t) values, and thus shared a similar source. Thus, the Carboniferous to Permian sediments received detritus across the PAO, indicating the closure of the PAO. Subsequently, the NAOB entered into an extensional setting based on the detrital zircon age patterns, rift-related volcanic rocks and basin analysis. Finally, a tectono-paleogeographic reconstruction from the Carboniferous relic sea and marine transgression to Permian marine regression-transgression-regression with crustal extension was proposed.