Grain shape is an important trait for improving rice yield. A number of quantitative trait loci (QTLs) for this trait have been identified by using primary F2 mapping populations and recombinant inbred lines, in which QTLs with a small effect are harder to detect than they would be in advanced generations. In this study, we developed two advanced mapping populations (chromosome segment substitution lines [CSSLs] and BC4F2 lines consisting of more than 2000 individuals) in the genetic backgrounds of two improved cultivars: a japonica cultivar (Koshihikari) with short, round grains, and an indica cultivar (IR64) with long, slender grains. We compared the ability of these materials to reveal QTLs for grain shape with that of an F2 population. Only 8 QTLs for grain length or grain width were detected in the F2 population, versus 47 in the CSSL population and 65 in the BC4F2 population. These results strongly suggest that advanced mapping populations can reveal QTLs for agronomic traits under complicated genetic control, and that DNA markers linked with the QTLs are useful for choosing superior allelic combinations to enhance grain shape in the Koshihikari and IR64 genetic backgrounds.