The desert steppe is unique as a transition zone from the steppe to the desert of the Eurasian Steppe. The spatial distribution of shrubs and perennial plants has changed due to long-term disturbance caused by overgrazing in the ecosystem. This study aimed to determine the degree of patchiness of shrubs and perennial plants under grazing disturbance, which is of great significance to better understand the succession mechanism of plant community in the desert steppe. This study investigated the spatial distribution of shrubs and perennial plants under four stocking rates (no grazing, CK, 0 sheep·hm−2·half year−1; light grazing, LG, 0.93 sheep·hm−2·half year−1; moderate grazing, MG, 1.82 sheep·hm−2·half year−1; heavy grazing, HG, 2.71 sheep·hm−2·half year−1) in the desert steppe of Inner Mongolia, primarily using geostatistical methods. The density of perennial grasses increased with increasing stocking rate, while its height decreased (except for HG). The density and height of the shrubs and the perennial forbs showed an exponential or gradual decline. The spatial distribution of shrubs and perennial plants is mainly affected by structural factors such as topography, soil parent material, and climate, and the density of shrubs and perennial plants is most sensitive to changes in stocking rate. An appropriate stocking rate increases the spatial heterogeneity of shrubs and perennial plants, but heavy grazing reduces the spatial heterogeneity of shrubs and perennial plants.