Quantitative trait loci (QTLs) for the apparent quality of brown rice under high temperatures during ripening were analyzed using chromosomal segment substitution lines. Segments from the indica cultivar Habataki were substituted into a japonica cultivar with a Sasanishiki background. We found the following two QTLs for increasing grain quality in the Habataki allele on chromosome 3: (1) qTW3-2, located near the marker RM14702, decreased the percentage of total white immature (TWI) grains, and (2) qRG3-2, located near RM3766, increased the percentage of regular grains. The effects of these two QTLs were more obvious under high-temperature ripening conditions; hence, these loci are considered QTLs not only for reducing TWI grains but also for increasing high-temperature tolerance. Additionally, we found two QTLs, i.e., qTW3-1 and qRG3-1, responsible for reduced grain quality near RM14314 on chromosome 3. Although the QTL for narrow grains in the Habataki allele qNG3 was genetically linked to qTW3-2, the effect was only slightly significant, and the length/width ratio of qNG3-carrying grains was within the range observed in widely grown japonica cultivars. Incorporating the Habataki region, including qRG3-2 and qTW3-2 but not qTW3-1 and qRG3-1, in addition to previously reported grain quality QTLs in breeding japonica cultivars will improve high-temperature tolerance and grain quality.