A better understanding of the underlying ecological mechanisms of diversity-biomass relationships in forest layers (i.e., overstory and understory) is critical to understand the importance of vertical stratification to the functioning of forest ecosystems. However, it is not clear how multiple abiotic (i.e., climate and geography) and biological (i.e., biodiversity, functional characteristics, and stand structural complexity) factors simultaneously determine the aboveground biomass (AGB) of each individual forest stratum. We used data on 156,270 trees from 1986 plots in North China to explore the relationships among biological diversity, plant functional traits, stand structure, climate and topography on variation in AGB of each stratum. The results showed that different biological factors determined the AGB of overstory and understory, and thus indicating different underlying ecological mechanisms in temperate forests. The effects of forest biodiversity on AGB were only significant in understory stratum. In the overstory of the forest, forests with high tree-size dimension inequality and high dominant tree height had larger AGB, hence mass ratio effect and stand structure complexity were the main ecological mechanisms for high biomass. In understory, diversity and overstory attributes were the main factors affecting biomass. Tree height and AGB of the overstory reduced the AGB of the understory layer. In consequence overstory attributes and niche complementation were the main ecological mechanisms in the understory. The overstory exerted influence on the understory through resource quantity and resource heterogeneity. Our findings have important implications for carbon management, enhancement of forest functions and sustainable forest management in temperate forests.