The mid-domain effect (MDE) explains altitudinal patterns of species diversity of mountainous plants at different elevations. However, its application is limited by the species life form and family flora in different layers of plant communities. To verify the MDE hypothesis at the plant community level, we chose a mountain with representative characteristics of the study area in the east of the Loess Plateau, China, such as obvious elevation (from 1324 to 2745 m) and latitude (from 36° 23′ to 39° 03′) gradients and considerable vegetation types (mainly coniferous and broad-leaved forests). We measured the life forms, families, and species diversity indices of tree, shrub, and herb communities along different elevations. We determined that the family numbers of the herb and shrub communities presented unimodal patterns across an altitudinal gradient, and the highest values occurred at intermediate elevations. The importance values of dominant families in the shrub and tree communities presented unimodal patterns, but the lowest values occurred at intermediate elevations. The species diversity indices of the herb, shrub, and tree communities conformed to unimodal change patterns following an altitudinal gradient, but the greatest diversity occurred at high, low, and intermediate elevations, respectively. At higher elevations, forbs and grasses grew well, whereas sedges grew well at lower elevations. Responses of different tree life forms to the altitudinal gradient were greater for evergreen coniferous tree species than for deciduous coniferous and deciduous broad-leaved tree species. We concluded that the MDE hypothesis of species diversity for mountainous plants is influenced greatly by the community life form and family at the plant community level in a temperate semi-arid region of the Loess Plateau, China. This conclusion tested and modified the MDE hypothesis and may be valuable for fueling prediction of biodiversity models and for the comparison with similar studies in arid and semi-arid mountainous regions.