Crop resistance to herbicides is crucial for agricultural productivity and sustainability amidst escalating challenges of weed resistance. Uncovering herbicide resistant genes is particularly important for rice production. In this study, we tested the resistance to three commonly used herbicides: glufosinate, glyphosate and mesotrione of 421 diverse rice cultivars and employed genome-wide association studies (GWAS) to unravel the genetic underpinnings of resistance to these three herbicides in rice. We discovered that cultivated rice exhibited rich variation in resistance to the three herbicides, and the differences among subpopulations were significant. Six identified associations harboring candidate genes for resistance to these herbicides were significant. Among them, RGlu6 and RGly8 were the major QTL for resistance to glufosinate and glyphosate, respectively. The favorable alleles of RGlu6 and RGly8 were primarily present in japonica cultivars that originated from Europe, highlighting the geographic and genetic diversity of herbicide resistance and emphasizing the localized selection pressures in European rice varieties. Moreover, our findings might suggest that traditional target genes may not contain tolerant alleles in nature, and alternative mechanisms with novel loci associated with resistance may work. By mapping the genes for herbicide resistance, our results may help develop new strategies to combat the dual challenges on effective weed management and herbicide sustainability.