Perennial trees must maintain stem growth throughout their entire lifespan to progressively increase in size as they age. The overarching question of the molecular mechanisms that govern stem perennial growth in trees remains largely unanswered. Here we deciphered the genetic architecture that underlies perennial growth trajectories using genome-wide association studies (GWAS) for measures of growth traits across years in a natural population of Populus tomentosa. By analyzing the stem growth trajectory, we identified PtoP4H9, encoding prolyl 4-hydroxylase 9, which is responsible for the natural variation in the growth rate of diameter at breast height (DBH) across years. Quantifying the dynamic genetic contribution of PtoP4H9 loci to stem growth showed that PtoP4H9 played a pivotal role in stem growth regulation. Spatiotemporal expression analysis showed that PtoP4H9 was highly expressed in cambium tissues of poplars of various ages. Overexpression and knockdown of PtoP4H9 revealed that it altered cell expansion to regulate cell wall modification and mechanical characteristics, thereby promoting stem growth in Populus. We showed that natural variation in PtoP4H9 occurred in a BASIC PENTACYSTEINE transcription factor PtoBPC1-binding promoter element controlling PtoP4H9 expression. The geographic distribution of PtoP4H9 allelic variation was consistent with the modes of selection among populations. Altogether, our study provides important genetic insights into dynamic stem growth in Populus, and we confirmed PtoP4H9 as a potential useful marker for breeding or genetic engineering of poplars.