The objective of this study was to evaluate various wheat supplementation levels on rumen microbiota and fermentation parameter in Tibetan sheep. A total of ninety ram with an average 12.37 ± 0.92 kg at the age of 2 months were randomly allocated to three treatments: 0% wheat diet (CW, N = 30), 10% wheat diet (LW, N = 30), and 15% wheat diet (HW, N = 30) on a dry matter basis. The experiment was conducted over a period of 127 days, including 7 days of adaption to the diets. Our results showed that sheep fed 10% wheat exhibited optimal average daily gain and feed gain ratio compared with HW group (p < 0.05). The serum alkaline phosphatase concentration was the lowest when fed the 10% wheat diet (p < 0.05), whereas serum aspartate aminotransferase concentration was the highest (p < 0.05). Both acetate and propionate increased with increase in dietary wheat ratio (p < 0.05), while a greater decrease in concentrations of NH3 -N was observed (p < 0.05). In rumen fluid, 3413 OTUs were obtained with 97% consistency. Phylum Firmicutes was the predominant bacteria and accounted for 49.04%. The CW groups supported significantly increased the abundance of Bacteroidetes (p < 0.05), as compared with the HW group. The abundance of Bacteroidales_UCG-001, Ruminococcus, and Mitsuokella possessed a higher relative abundance in HW group (p < 0.05). No differences in the bacterial community and fermentation parameters were observed between the sheep fed 0% and 10% wheat (p > 0.05). Ruminal bacterial community structure was significantly correlated with isobutyrite (r2 = 0.4878, p = 0.035) and valerate (r2 = 0.4878, p = 0.013). In conclusion, supplementation of 10% wheat in diet promoted the average daily gain and never altered microbial community structure and fermentation pattern, which can be effectively replace partial corn in Chinese Tibetan Sheep.