Several key genes governing Zn homeostasis and grain zinc content (GZC) have been functionally characterized. However, the effects of these genes in diverse breeding populations have not been evaluated; thus, their availability in breeding is unclear. In this study, the effects of 65 genes related to rice zinc responses on GZC were evaluated using two panels of breeding lines, and the superior haplotypes were identified. One panel consisted of mega varieties from the International Rice Research Institute (IRRI), South Asia, and Southeast Asia (SEA), and the other panel is breeding lines/varieties from South China (SC). In addition, a multiparent advanced generation intercross (MAGIC) population, named as DC1, was also employed. Three analytical methods, single-locus mixed linear model (SL-MLM), multilocus random-SNP-effect mixed linear model (mrMLM), and haplotype-based association analysis (Hap-AA), were applied. OsIDEF1 (which explained 12.3% of the phenotypic variance) and OsZIFL7 (8.3–9.1%), OsZIP7 (18.9%), and OsIRT1 (17.9%) were identified by SL-MLM in SEA and SC, respectively, whereas no gene was significantly associated with GZC in DC1. In total, five (OsNRAMP6, OsYSL15, OsIRT1, OsIDEF1, and OsZIFL7, 7.70–15.39%), three (OsFRDL1, OsIRT1, and OsZIP7, 11.87–17.99%), and two (OsYSL7 and OsZIP7, 9.85–10.57%) genes were detected to be significantly associated with GZC in SEA, SC, and DC1 by mrMLM, respectively. Hap-AA indicated that Hap1-OsNRAMP5, Hap5-OsZIP4, Hap1-OsIRT1, Hap3-OsNRAMP6, Hap6-OsMTP1, and Hap6-OsYSL15 had the largest effects for GZC in SEA, whereas Hap3-OsOPT7, Hap4-OsIRT2, Hap4-OsZIP7, Hap5-OsIRT1, and Hap5-OsSAMS1 were the most significant in the SC population. Besides, superior alleles were also identified for the significant genes. The genes significantly associated with GZC and their superior haplotypes identified in different panels could be used in enhancing GZC through molecular breeding, which could further address the problem of Zn malnutrition among rice consumers.