Knowledge of the tectonic history of the North Qilian Shan contributes to our understanding of the formation of the Tibetan Plateau. However, when the North Qilian Shan were formed has been controversial. We report new detrital zircon U-Pb (n = 899) and double dating fission track and U-Pb (n = 450) results from nine samples of Lower Cretaceous and Oligocene sandstone units, integrating with published geochronology and thermochronology data in order to reconstruct the evolution history of the North Qilian Shan. Nine samples from this unit show detrital zircon fission track age populations that center at: 636–603, 495, 406–331, 290–266, 247–224, 193–151, and 144–102 Ma. These fission track data are older than their corresponding depositional ages, suggesting that the fission track ages are unreset and record magmatic-related cooling or exhumation stages of the North Qilian Shan since the Late Neoproterozoic. Deconvolved fission track peak ages document major cooling events associated with the opening of the North Qilian Ocean at 636–603 Ma. Late Cambrian peak age (495 Ma) is attributed to the early phase of the subduction of the North Qilian Ocean. The Devonian period exhibited a cooling phase between 406 and 381 Ma, corresponding to the collision event between the Qaidam terrane, Central Qilian Shan, and Alxa Block. The peak age of 331 Ma signifies the cooling event associated with the subduction of the Paleo-Asian Ocean. The peak ages during the Early-Middle Permian period (290–266 Ma) provide evidence of significant exhumation events that occurred during the Late Paleozoic era. These events were linked to the subduction of the Paleo-Tethys Ocean to the south and the Paleo-Asian Ocean to the north. The Triassic peak ages (247–224 Ma) unveil exhumation signals in the North Qilian Shan region that were generated by the far-field effect of the collision between the Qiangtang and Kunlun terranes. During the Jurassic (193–151 Ma) and the Early Cretaceous periods (144–102 Ma), the exhumation of the North Qilian Shan region was a consequence of the subduction of the Neo-Tethys Ocean and the collision between the Lhasa and Qiangtang blocks, respectively. The peak ages of the Xinminpu Formation in the Early Cretaceous (123–121 Ma) are in close proximity to their depositional age (125–105 Ma), indicating that the North Qilian Shan underwent rapid exhumation during this period. The peak ages observed in the Oligocene Baiyanghe Formation (31–24 Ma) from 144 to 102 Ma, indicate that it has been influenced by the recycled sediments originating from the Cretaceous Xinminpu Formation. The North Qilian Shan gradually emerged during the late Paleozoic and Mesozoic eras, providing the foundation for its subsequent reactivation in the Cenozoic era. The multi-stage cooling or exhumation events of North Qilian Shan since Late Neoproterozoic reflect the complex formation process of the northeastern margin of Tibetan Plateau.