The neighborhood effect has become an important framework with which to study the mechanisms that maintain the coexistence of tree species. Phylogenetic relatedness among neighboring plants directly affects species coexistence and the maintenance of tree diversity. And some studies have reported that seedling performance is negatively correlated with phylogenetic relatedness, which termed phylogenetic negative density dependence. Soil-borne fungal pathogens affected seedling performance of phylogenetically related host species, i.e., phylogenetic Janzen–Connell effect. Seedlings may be particularly vulnerable to habitat and neighbor characteristics. Although previous studies have demonstrated the influence of neighborhood effects, phylogenetic relatedness, and habitat filtering on seedling survival, growth, and mortality, the effect of variation in these factors on seedling abundance remains unclear. To address this question, we used a 4-ha (200 m × 200 m) and monitored four-year (2020–2023) seedling dataset from a mid-montane humid evergreen broad-leaved subtropical forest in the Gaoligong Mountains, Yunnan, Southwestern China, and which consisted of 916 seedlings belonging to 56 species. The results of generalized linear mixed models showed no significant effect of conspecific adult neighbors on seedling abundance at any of the intervals evaluated. In contrast, we found evidence of phylogenetic distance density dependence in the forests of the Gaoligong Mountains. Specifically, there was a significant positive effect of the relative average phylogenetic distance between heterospecific adult neighbors and focal seedlings on focal seedling abundance in 2020; however, the relative average phylogenetic distance between heterospecific seedling neighbors and focal seedlings had a significant negative effect on seedling abundance over the four-year period (2020–2023). Among the habitat factors, only light (canopy opening) had a negative effect on seedling abundance in all four years. Light resources may be a limiting factor for seedlings, and determine seedling dynamics in subtropical forests. Overall, our results demonstrated that phylogenetic density dependence and habitat filtering affected subtropical seedling abundance. Our findings provide new evidence of the impact of phylogenetic density dependence on seedling abundance in a subtropical mid-montane humid evergreen broad-leaved forest and highlight the need to incorporate the neighborhood effect, phylogenetic relatedness, and habitat factors in models assessing seedling abundance.