A total of 194 groundwater samples were collected from wells in hard rock aquifers of the Medak district, South India, to assess the distribution of fluoride in groundwater and to determine whether this chemical constituent was likely to be causing adverse health effects on groundwater user in the region. The study revealed that the fluoride concentration in groundwater ranged between 0.2 and 7.4 mg/L with an average concentration of 2.7 mg/L. About 57% of groundwater tested has fluoride concentrations more than the maximum permissible limit of 1.5 mg/L. The highest concentrations of fluoride were measured in groundwater in the north-eastern part of the Medak region especially in the Siddipeta, Chinnakodur, Nanganoor and Dubhaka regions. The areas are underlain by granites which contain fluoride-bearing minerals like apatite and biotite. Due to water–rock interactions, the fluoride has become enriched in groundwater due to the weathering and leaching of fluoride-bearing minerals. The pH and bicarbonate concentrations of the groundwater are varied from 6.6 to 8.8 and 18 to 527 mg/L, respectively. High fluoride concentration in the groundwater of the study area is observed when pH and the bicarbonate concentration are high. Data plotted in Gibbs diagram show that all groundwater samples fall under rock weathering dominance group with a trend towards the evaporation dominance category. An assessment of the chemical composition of groundwater reveals that most of the groundwater samples have compositions of Ca2+–Mg2+–Cl− > Ca2+–Na+–HCO3 − > Ca2+–HCO3 − > Na+–HCO3 −. This suggests that the characteristics of the groundwater flow regime, long residence time and the extent of groundwater interaction with rocks are the major factors that influence the concentration of fluoride. It is advised not to utilize the groundwater for drinking purpose in the areas delineated, and they should depend on alternate safe source.