AbstractThe formation age of the middle Yellow River and the existence of a northward‐flowing river have been fiercely debated. The age distribution of detrital zircon varied spatiotemporally and produced contradictory provenance interpretations. The Jinshaan Gorge, the main part of the middle Yellow River and key to studying fluvial evolution and clarifying disputes, developed its topography during the late Cenozoic. In this study, we systematically review the Cenozoic tectonic evolution of the North China Craton, perform detrital zircon U–Pb dating in the Neogene−Quaternary sediments and investigate the topography along the Jinshaan Gorge, and the sedimentology and chronological framework of these sediments. We propose that the Gorge of the middle Yellow River could have developed since the Neogene, controlled by the tectono‐geomorphologic evolution of the North China Craton in a dominantly extensional environment. No evidence supports a northward‐flowing river during the Early Pleistocene or even earlier in the Jinshaan Gorge. We attribute the provenance variations of the Cenozoic sediments to detrital mixing of diverse geological units, local and distant, and especially highlight the systematic provenance shift between the Neogene and Quaternary sediments caused by bedrock downcutting and recycling aeolian sediments. The increased 1.5−0.33 Ga component of the lower Yellow River during the Early Pleistocene was likely caused by enhanced loess accumulation and should not be individually used as a proxy for the Yellow River formation. We emphasize the significance of a comprehensive study of river evolution.