The forcing mechanism behind river incision and terrace formation is one of hot topics in the study of fluvial geomorphology. This work focused on the late Quaternary alluvial sequence in the south piedmont of the Chinese Altay Shan in the arid inland of Asia. In order to reveal the mechanism controlling the development of late Quaternary fluvial features along the mountain front, we conducted detailed fluvial geomorphological investigations on eight rivers, including geomorphic mapping, optically stimulated luminescence (OSL) dating, and differential global position system (dGPS) surveying. By utilizing the Monte Carlo simulation with terrace data, the late Quaternary rates of river incision along the south piedmont of the Chinese Altay Shan were determined. The results show that (1) the terraces developed by the piedmont rivers are not more than four levels and the depth of river incision in the piedmont is only dozens of meters, with deeper valleys displayed by the western piedmont rivers; (2) terrace alluviums along the mountain front were accumulated during glacial stages, and the subsequent incision occurred during interglacial stages; (3) the rate of river incision was accelerated during the Holocene, when the regional climate was characterized by progressively increasing wetness. When combining with the regional tectonic setting, we propose that the climate could have played the key role in driving the alluvial accumulation and the subsequent incision in the south piedmont of the Chinese Altay Shan during the late Quaternary. Together with the similar observations from the northeastern margin of the Tibetan Plateau and the Tian Shan, we further propose that climatically-driven fluvial geomorphological development could be a common phenomenon during the late Quaternary in the arid interior of Asia.