The experiment was to compare the effects of switching all-concentrate to all-roughage diets on rumen microflora and functional metabolism of yak, cattle-yak, Tibetan yellow cattle and yellow cattle living in different altitudes. A total of 24 yaks, cattle-yaks, Tibetan yellow cattle and yellow cattle with a similar weight and good body condition aged 3.5 years were selected and divided into four groups according to species. They were fed a concentrate diet with 40% soybean meal and 60% corn meal for the first month (C group) and a roughage diet with dry corn stalks (100%) for the second month (R group); the formal experimental period was 60 d. These results showed that the conversion had a significant effect on the rumen microflora structure of the four herds, and the biggest difference between concentrate and roughage diets was yak and cattle-yak, followed by Tibetan yellow cattle and yellow cattle. At the phylum level, Bacteroidetes and Firmicutes still predominate in all groups. Compared with the C groups, the relative abundance of Lentisphaerae and Kiritimatiellaeota increased in all R groups, and Lentisphaerae was significantly increased in yak and cattle-yak (p < 0.05). At the genus and species levels, Prevotella had the highest abundance, and the relative abundances of Prevotella, Ruminococcus, Sarcina and Ruminobacter in R groups were lower, while the abundances of other differential genera, including Methanobrevibacter, Fibrobacter, Treponema, Eubacterium, Butyrivibrio, Succinivibrio and Succinimonas, were all higher. Roughage diets increased the number of unique genes and functional genes encoding different CAZymes in rumen microorganisms in all four herds. In the functional contribution analysis, with the exception of ABC transporters and methane metabolism, Prevotella was the main contributor to almost all of these functions. In methane metabolism, Methanobrevibacter had the highest relative abundance, followed by Prevotella, Clostridia and Bacteroidales in all groups. Compared with Tibetan yellow cattle and yellow cattle, yaks and cattle-yaks have better adaptability to roughage, and its utilization rate can be fully improved to reduce methane emission. The study indicates that when four herds are converted to high roughage at the later stage of feeding, the growth and reproduction of rumen microorganisms are affected, and the abundance and diversity of rumen microorganisms are increased to varying degrees. The transformation of concentrate to roughage diet can change the metabolic pathways of rumen microorganisms in yaks and finally affect the fermentation mode of rumen. The above results provide a theoretical basis for the research and development of fattening feeds for yaks, cattle-yaks, Tibetan yellow cattle and yellow cattle and the intensive feeding of livestock on the plateau.