An increasing number of phylogeographic studies have been conducted for plant species in the Qinghai-Tibetan Plateau (QTP) and its flanking mountains. However, these studies have mainly focused on the determination of glacial refugia and routes of inter-/post-glacial expansions. Rapid intraspecific diversification of plants in this region have not been thoroughly discussed. Herein, we investigate the effects of the Quaternary climate changes on population genetic structure and diversifications of a herbaceous alpine species, Saxifraga sinomontana, which may have an evolutionary time scale <5 million years in the QTP and Himalayan regions. Using a total of 350 individuals from 29 populations, we studied the evolutionary history of S. sinomontana by analyzing cpDNA trnL-trnF, rpl16 and nrDNA ITS sequences. A total of 89 haplotypes and 158 genotypes were detected for cpDNA and ITS sequences, respectively. Only a few haplotypes/genotypes were widespread, while an extremely large number of haplotypes/genotypes were restricted to single populations, which were scattered throughout the current geographical range of S. sinomontana. This suggests the existence of microrefugia of this species during the Quaternary glaciations. In addition, the relationships of the haplotypes/genotypes were almost completely not resolved by phylogenetic reconstruction. Combining characteristics in terms of high haplotype richness, large proportion of private haplotypes, and shallow haplotype divergence, we speculate that recent intraspecific diversification has occurred in S. sinomontana. Molecular clock analysis estimated that the onset diversification within S. sinomontana to be 1.09 Ma (95% HPD = 0.80–1.45), coinciding with the extensive Quaternary glaciations on the QTP which started ca. 1.17 Ma. The Quaternary climatic oscillations may have triggered rapid intraspecific diversification in this QTP-Himalayan species. However, large niche breadth, as well as introgression/hybridization between the studied species and its closely related sympatric saxifrages, may also played a role to some extent on the current genetic structure of S. sinomontana, which need to be further studied.