Excessive fluoride ion (F−) concentrations in mine water affected by the geologic processes and coal mine activity prevent the reusage of it as domestic and production water resources. There are few researches on the accumulation mechanism of fluoride in mine water. In this study, 71 water samples and 39 rock samples were collected from 9 mines of the Inner Shaanxi-Mongolia Contiguous area for chemical analyses and laboratory experiments. Results indicate that the concentration of F− in mine water increased with the gradual increase of the coal mining depth. The average concentration of F− in no.3-1 coal mine water is higher than that in no.2-2 coal mine water. And concentrations of F− exceed the China national standard for drinking water quality (upper limit of 1 mg/L), which is closely related to fluoride in the no.3-1 coal roof groundwater and water-rock interaction in the goaf. The main hydrogeochemical processes include mineral dissolution (fluorapatite and hornblende), desorption of exchangeable F− from clay minerals (mica, illite, kaolinite and chlorite) and cation exchange between Ca2+ and Na+. Furthermore, long-term further water–rock interactions result in F− accumulation in mine water. Alkaline environments, decreased particle size of rock, and increased temperature all favor release of F− from clay mineral surface into mine water. Finally, conceptual model of high-fluoride mine water was first proposed, and the formation of high-fluoride mine water can be summed as two stages including water-rock interaction in groundwater and secondary water-rock interaction in goaf. This study helps to improve our understanding of genesis of high-fluoride mine water and our capability of predicting the future concentration, which may provide foundation for manage and utilization of mine water.