We undertook LA-ICP-MS U–Pb dating of detrital zircons from Late Paleozoic sedimentary rocks in eastern Heilongjiang Province, NE China, with the aim of constraining the tectonic evolution of the eastern segment of the Central Asian Orogenic Belt (CAOB). Detrital zircons from quartz sandstone of the Devonian Heilonggong Formation in the Zhangguangcai Range yield age populations of ∼ 2503, ∼ 1833, ∼ 903, ∼ 802, and 551–403 Ma. Zircons from quartz sandstone of the Devonian Baoquan Formation in the Lesser Xing'an Range yield age clusters at ∼ 2500, ∼ 800, and 489–425 Ma. These ages reveal that: (1) rocks from the Heilonggong and Baoquan formations were deposited after 391 and 413 Ma, respectively; (2) these rocks were sourced from regions with rocks as old as ∼ 2500 Ma, with dominant age populations of ∼ 2500, ∼ 1833, 903–800, and 551–403 Ma; and (3) in addition to new lower crust accreted during the Phanerozoic (as revealed by voluminous Phanerozoic granitoids with positive εNd values within the CAOB), it appears that several remnants of an ancient Precambrian basement occur adjacent to these Late Paleozoic basins (e.g., the Shangzhi, Tangyuan, and Baoqing basins) that occur within microcontinental massifs including the Songnen–Zhangguangcai Range, Xing'an, and Erguna massifs. Detrital zircons from quartz sandstone of the Devonian Heitai Formation on the eastern margin of the Jiamusi Massif yield peak age populations of 569, 542, 509, and 484 Ma, similar to those obtained from zircons within the Mashan Group of the Jiamusi Massif. These age data indicate that: (1) deposition of the Heitai Formation on the eastern margin of the Jiamusi Massif had started by 484 Ma, and (2) these sediments were sourced entirely from the Jiamusi Massif. The widespread occurrence of detrital zircons with ages of 480–550 Ma within Late Paleozoic basins in the Lesser Xing'an–Zhangguangcai Ranges and the Jiamusi Massif, combined with the presence of a Caledonian (445–499 Ma) I-type granitoid belt in the Lesser Xing'an–Zhangguangcai Ranges, indicates that amalgamation of the Songnen–Zhangguangcai Range and Jiamusi massifs took place before the Early Devonian, probably during the late stage of the Early Paleozoic (after ∼ 445 Ma).