Bongo and Sekyere South districts, both in the northern and southern parts of Ghana, respectively, have high populations living in rural areas and most of them use groundwater for drinking purposes. The groundwater in these areas is prone to contamination from natural and/or artificial sources. Therefore this study aims; (1) to present a comparative analysis of the fluoride concentration in groundwater samples from Bongo and Sekyere South districts and the associated groundwater–rock interaction that may be the cause for the varied fluoride concentrations, (2) to determine the leaching potential of fluoride from the host rocks as the possible mechanism for groundwater contamination. Sixty (60) groundwater samples from active pumping wells and twelve (12) rock samples from outcrops were collected from various communities in the two districts for fluoride concentration and mineralogical analysis. Based on the variations in fluoride concentration, fluoride spatial distribution maps were prepared using empirical Bayesian kriging interpolation method and analysed by means of hierarchical cluster analysis. The fluoride concentration in Bongo district varies between 1.71 and 4.0 mg/L, whereas that in Sekyere South district changes from 0.3 to 0.8 mg/L. From the mineralogical studies, biotite has the highest percentage in the Bongo district and has positive correlation with fluoride concentration in the analysed water samples than in the Sekyere South district. The elevated fluoride concentration in the Bongo district relative to the Sekyere South district is due to the dissolution of biotite in the groundwater and the sufficient groundwater–rock interaction since the water samples are mainly sourced from deeper boreholes. This high fluoride concentration has resulted in a plethora of reported cases of dental fluorosis and other health-related issues in Bongo.