The vertical distribution understanding of undergrowth herbaceous plants in high altitude mountain areas is of great significance to reveal the environmental gradient change law of plant diversity. However, the species composition of herbaceous plants at different altitude gradients and the relationship between diversity and environmental factors is still vague. This study takes the Sejila Mountain in southeast Tibet at an altitude of 2,000–3,300 m as the study area to analyze the vertical distribution differences of the understory herbs and their driving factors on shady and sunny slopes. The results showed that: (1) The number of families, genera and species of herbaceous plants on sunny slope is greater than that on shady slope. Asteraceae on shady and sunny slopes has the largest number of species; (2) The α diversity index of the herbs on shady slope shows a U-shaped trend. The Cody heterogeneity index (βC) is negatively correlated with the β diversity index. The herb richness index (R) on sunny slope is unimodal. The β diversity index displays the same as that of shady slope; (3) Different environmental factors have different effects on herb diversity. The correlation between environmental factors and diversity indicators indicates that there is a certain degree of clustering among samples. The environmental factors of shady slope and sunny slope are separated; and (4) The difference of water and heat conditions caused by altitude and slope aspect is an important driving factor for the diversity of herbage species, followed by soil clay content, silt content and slope gradient. This study will help to understand the species diversity and ecosystem characteristics of the region, and provide a reference for the study of plant community structure and function and rational utilization of understory plant resources in high-altitude mountain areas.