AbstractThe size structure transition matrices of Picea jezoensis, Picea glehnii and Abies sachalinensis of a sub‐boreal forest in Hokkaido, northern Japan were constructed based on the demography of each species (Picea jezoensis and Picea glehnii were dealt with together as Picea) during a 4‐year period. Two types of matrices, density‐independent and density‐dependent population dynamics models, were investigated for evaluating the ‘waiting pattern’ between Picea spp. and A. sachalinensis. For the density‐dependent model, it was assumed that the demographic traits of understory trees, the recruitment rate, the understory mortality rate and the transition probability from the understory to canopy stages, were regulated by the one‐sided competitive effect of canopy trees. The observed size structure of Picea was almost consistent with the stationary size structure obtained in both the density‐independent and the density‐dependent models, whereas the observed size structure of A. sachalinensis was not realized in the two models. The effects of both the transition probability from the understory to canopy stages and the recruitment rate on the dynamics of canopy trees were investigated. For Picea, two parameters—recruitment rate (ei) and transition probability from the understory to canopy stages‐exponentially affected the dynamics of canopy trees. In contrast, for Abies sachalinensis, the two parameters affected linearly the dynamics of canopy trees. In conclusion, the population dynamics of Picea and A. sachalinensis was determined by the parameters of the recruitment rate and the transition probability from the understory to canopy stages, relating to waiting patterns of understory trees for future gap formation. In Picea, the demographic parameters of understory trees intensively regulated the dynamics of canopy trees if compared with A. sachalinensis, suggesting that the performance of understory trees plays a key role in the population dynamics of Picea. This reflects the growth pattern of understory trees in the regeneration of the two species.