Heterophylly, the existence of different leaf shapes and sizes on the same plant, has been observed in many flowering plant species. Yet, the genetic characteristics and genetic basis of heterophylly in soybean remain unknown. Here, two populations of recombinant inbred lines (RILs) with distinctly different leaf shapes were used to identify loci controlling heterophylly in two environments. The ratio of apical leaf shape (LSUP) to basal leaf shape (LSDOWN) at the reproductive growth stage (RLS) was used as a parameter for classifying heterophylly. A total of eight QTL were detected for RLS between the two populations and four of them were stably identified in both environments. Among them, qRLS20 had the largest effect in the JS population, with a maximum LOD value of 46.9 explaining up to 47.2% of phenotypic variance. This locus was located in the same genomic region as the basal leaf shape QTL qLSDOWN20 on chromosome 20. The locus qRLS19 had the largest effect in the JJ population, with a maximum LOD value of 15.2 explaining up to 27.0% of phenotypic variance. This locus was located in the same genomic region as the apical leaf shape QTL qLSUP19 on chromosome 19. Four candidate genes for heterophylly were identified based on sequence differences among the three parents of the two mapping populations, RT-qPCR analysis, and gene functional annotation analysis. The QTL and candidate genes detected in this study lay a foundation for further understanding the genetic mechanism of heterophylly and are invaluable in marker-assisted breeding.