Smallholder maize farming systems in sub-Saharan Africa (SSA) are vulnerable to drought-induced yield losses, which significantly impact food security and livelihoods within these communities. Mapping and characterizing genomic regions associated with water stress tolerance in tropical maize is essential for future breeding initiatives targeting this region. In this study, three biparental F3 populations composed of 753 families were evaluated in Kenya and Zimbabwe and genotyped with high-density single nucleotide polymorphism (SNP) markers. Quantitative trait loci maping was performed on these genotypes to dissect the genetic architecture for grain yield (GY), plant height (PH), ear height (EH) and anthesis-silking interval (ASI) under well-watered (WW) and water-stressed (WS) conditions. Across the studied maize populations, mean GY exhibited a range of 4.55–8.55 t/ha under WW and 1.29–5.59 t/ha under WS, reflecting a 31–59% reduction range under WS conditions. Genotypic and genotype-by-environment (G × E) variances were significant for all traits except ASI. Overall broad sense heritabilities for GY were low to high (0.25–0.60). For GY, these genetic parameters were decreased under WS conditions. Linkage mapping revealed a significant difference in the number of QTLs detected, with 93 identified under WW conditions and 41 under WS conditions. These QTLs were distributed across all maize chromosomes. For GY, eight and two major effect QTLs (>10% phenotypic variation explained) were detected under WW and WS conditions, respectively. Under WS conditions, Joint Linkage Association Mapping (JLAM) identified several QTLs with minor effects for GY and revealed genomic region overlaps in the studied populations. Across the studied water regimes, five-fold cross-validation showed moderate to high prediction accuracies (−0.15–0.90) for GY and other agronomic traits. Our findings demonstrate the polygenic nature of WS tolerance and highlights the immense potential of using genomic selection in improving genetic gain in maize breeding.