Microbial activity in stored rice grains could cause quality deterioration and mycotoxin accumulation that may lead to serious economic losses and food safety risks. However, limited studies have examined the spatial and temporal variation of microbial communities associated with stored rice grains. Here, we performed amplicon sequence analysis to investigate the temporal and spatial distribution of microbes in stored rice grains from Chongqing grain depot (Southern China) and Liaoning grain depot (Northern China). Bacterial and fungal diversities, in addition to community structures of rice grains in Chongqing were significantly different from those in Liaoning in terms of α diversity (Chongqing > Liaoning, p <001) and β diversity (p < 0.001, bacterial communities: R = 0.9293, fungal communities: R = 1.0). The core bacterial taxa among stored rice grains comprised Pantoea, Pseudomonas, Rhizobium, and Methylobacterium, while the core fungal taxa comprised Alternaria, Aspergillus, and Cladosporium. In addition, different microbial communities were observed at different stored time points (i.e., post-harvest period, storage for one year, and storage for two years), and along different stored vertical depths (upper, middle, and lower layers) within the Chinese horizontal warehouse. The relative abundances of Aspergillus increased over storage time, but decreased with the increase of stored vertical depth within the warehouse. Temperature was the most important factor associated with differences in microbial communities across storage periods. The potential mycotoxin producing fungal species A. flavus and A. niger exhibited significantly higher abundances in rice grains of Chongqing compared to those of Liaoning. These data could be useful for evaluating potential risks of toxigenic fungi and mycotoxin contamination of stored rice grains. Further, these insights can help grain depot managers optimize storage conditions and reduce risks of deleterious fungi during rice grains storage.