Decision-making related to nitrogen (N) fertilization is a crucial step in agronomic practices because of its direct interactions with agronomic productivity and environmental risk. Here, we hypothesized that soil apparent N balance could be used as an indicator to determine the thresholds of N input through analyzing the responses of the yield and N loss to N balance. Based on the observations from 951 field experiments conducted in rice (Oryza sativa L.) cropping systems of China, we established the relationships between N balance and ammonia (NH3) volatilization, yield increase ratio, and N application rate, respectively. Dramatical increase of NH3 volatilizations and stagnant increase of the rice yields were observed when the N surplus exceeded certain levels. Using a piecewise regression method, the seasonal upper limits of N surplus were determined as 44.3 and 90.9 kg N ha−1 under straw-return and straw-removal scenarios, respectively, derived from the responses of NH3 volatilization, and were determined as 53.0–74.9 and 97.9–112.0 kg N ha−1 under straw-return and straw-removal scenarios, respectively, derived from the maximum-yield consideration. Based on the upper limits of N surplus, the thresholds of N application rate suggested to be applied in single, middle-MLYR, middle-SW, early, and late rice types ranged 179.0–214.9 kg N ha−1 in order to restrict the NH3 volatilization, and ranged 193.3–249.8 kg N ha−1 in order to achieve the maximum yields. If rice straw was returned to fields, on average, the thresholds of N application rate could be theoretically decreased by 17.5 kg N ha−1. This study provides a robust reference for restricting the N surplus and the synthetic fertilizer N input in rice fields, which will guide yield goals and environmental protection.