Shrubs are widely distributed in semi-arid and arid ecosystems. Understanding shrub effects on alpha and beta diversity of soil fauna in arid and semiarid ecosystems could lead to promotion of biodiversity conservation and sustainable management. Yet the extent to which shrubs can influence beta diversity of soil fauna and different body size in arid and semiarid ecosystems is unknown. An investigation of soil fauna, macrofauna, and mesofauna beneath shrub canopy and in open spaces was conducted in the Donddashan Nature Reserve (a semiarid grassland ecosystem in China). Shrubs significantly increased the alpha diversity (richness and Shannon index), but significantly decreased the beta diversity and the nestedness component. Moreover, shrub effects differed among soil fauna of different body size. Shrubs significantly increased the abundance and Shannon index of mesofauna, whereas there were no significant effects on macrofauna between shrubs and open spaces. Shrubs significantly decreased the beta diversity of mesofauna, but significantly increased beta diversity and the turnover component of macrofauna. Piecewise structural equation modelling (SEM) showed that shrubs had an indirect effect on soil fauna diversity through mesofauna diversity. Results highlighted the crucial role that shrubs have in maintaining alpha diversity of soil fauna in arid ecosystem, and also demonstrated the divergent shrub effects on beta diversity of soil fauna, mesofauna, and macrofauna. This suggests a need to examine different taxonomic groups to gain a better understanding of ecological consequences of shrubs.