The objectives of this study were to detect QTLs for panicle number and grain yield in rice ( Oryza sativa L.) under low nitrogen and low phosphorus conditions and to analyze the genetic basis of tolerance to the soil nutrient deficiency. A set of 125 chromosome segment substitution lines (CSSLs), each containing a single or a few introgression segments from a japonica cultivar Nipponbare within the genetic background of an indica cultivar 9311, were evaluated using augmented design in field experiments with normal fertilization (NF), low nitrogen (NL), and low phosphorus (PL) treatments. Grain yield and panicle number per plant were measured for each CSSL, and their relative values based on NF treatment were considered as the measurement for tolerance to the nutrient deficiency. Both NL and PL treatments had strong negative effects on grain yield and panicle number. There were different responses among the CSSLs to the deficiency of nitrogen or phosphorus. The relative traits had a significantly negative correlation with the traits in NF treatment. Cultivar 9311 showed higher tolerance to low-nutrient stresses than Nipponbare. A total of 38 chromosomal regions or quantitative trait loci (QTLs) with negative allelic effects from Nipponbare were detected under the stresses of nitrogen and phosphorus deficiencies. Among them, 26 QTLs were responsible for yield and panicle number, and the remaining 12 QTLs specifying the relative traits. Five chromosomal regions were identified in common under both stresses. Most QTLs (81%) were specifically detected only in low nitrogen or phosphorus condition. These different QTLs suggest that the responses to limiting nitrogen and phosphorus conditions are regulated by different sets of genes in rice. Most QTLs for the relative traits were colocalized with those for the yield and the panicle number under either nitrogen deficiency or phosphorus deficiency stress, indicating that the tolerant QTLs may be involved in nitrogen and phosphorus uptake or assimilation pathway in rice.