Reducing nitrogen (N) fertilizer input into the soil is needed by a crop production and environmental pollution control. A field experiment using a wheat-maize rotation system was conducted in North China Plain (NPL) to evaluate agronomic performance and the reduction of nitrate accumulation. The trial consisted of three replicates of five treatments: no nitrogen (CK), recommended N rate (REC) (180 and 210 kg ha−1 N fertilizer as urea for wheat and maize, respectively), same amount N in the form of controlled release fertilizer (CRF), with addition of duck manure to achieve the same total N rate as REC (80% REC + DM) and conventional fertilization (CF) (315 and 270 kg ha−1 N fertilizer as urea for wheat and maize, respectively). During the continued fertilization of the rotation system, the CRF application had an equal yield, N use efficiency (48%) and residual N (175 kg ha−1) but decreased the estimated N loss (18 kg ha-1) when compared to REC (72 kg ha-1). N accumulated below the root zone in the 40–60 cm soil layer was at a high risk of migrating deeper in the soil profile. Application of CRF could effectively reduce the nitrate N accumulating in the soil, slowing down the rate of nitrate migration to the deep soil.