This study aimed to characterize the fungal diversity and mycotoxin concentrations of native grass hay in various storage periods. In the present study, the native grass hay samples were collected when stored for 0 d (D0 group), 30 d (D30 group), and 150 d (D150 group), respectively. Here, mycotoxin analyses combined with ITS gene sequence were performed to reveal the changes in response to the storage period. There were notable differences in deoxynivalenol and aflatoxin B1 concentrations among the three groups. Compared to the D150 group, the diversity of the fungal community was higher in the D0 and D30 groups, which indicating the diversity was significantly influenced by the storage period. No significant (p > 0.05) difference was observed among the three groups on the dominant phyla. Interestingly, a significant (p < 0.05) difference was also observed in Chactomella and Aspergillus among the three groups, the abundance of the Chactomella was significantly (p < 0.05) decreased and the abundance of Aspergillus was statistically (p < 0.05) increased in the D150 group. Correlation analysis of the association of fungi with mycotoxin could provide a comprehensive understanding of the structure and function of the fungal community. These results indicated that the good practices of storage are essential for the prevention of mycotoxin. The information contained in the present study is vital for the further development of strategies for hay storage with high quality in the harsh Mongolian Plateau ecosystem.