Maternal exposure to environmental pollutants could affect fetal brain development and increase autism spectrum disorder (ASD) risk in conjunction with differential genetic susceptibility. Organohalogen congeners measured in maternal midpregnancy blood samples have recently shown significant, but negative associations with offspring ASD outcome. We report the first large-scale maternal and fetal genetic study of the midpregnancy serum levels of a set of 21 organohalogens in a subset of 790 genotyped women and 764 children collected in California by the Early Markers for Autism (EMA) Project. Levels of PCB (polychlorinated biphenyl) and PBDE (polybrominated diphenyl ether) congeners showed high maternal and fetal estimated SNP-based heritability (h2g) accounting for 39–99% of the total variance. Genome-wide association analyses identified significant maternal loci for p,p′-DDE (P = 7.8 × 10−11) in the CYP2B6 gene and for BDE-28 (P = 3.2 × 10−8) near the SH3GL2 gene, both involved in xenobiotic and lipid metabolism. Fetal genetic loci contributed to the levels of BDE-100 (P = 4.6 × 10−8) and PCB187 (P = 2.8 × 10−8), near the potential metabolic genes LOXHD1 and PTPRD, previously implicated in neurodevelopment. Negative associations were observed for BDE-100, BDE153, and the sum of PBDEs with ASD, partly explained by genome-wide additive genetic effects that predicted PBDE levels. Our results support genetic control of midgestational biomarkers for environmental exposures by nonoverlapping maternal and fetal genetic determinants, suggesting that future studies of environmental risk factors should take genetic variation into consideration. The independent influence of fetal genetics supports previous hypotheses that fetal genotypes expressed in placenta can influence maternal physiology and the transplacental transfer of organohalogens.