Enhancing crop production by maintaining a proper synchrony between soil nitrogen (N) and crop N demand remains a challenge, especially in under-studied tropical soils of Sub-Saharan Africa (SSA). For two consecutive cropping seasons (2013–2015), we monitored the fluctuation of soil inorganic N and its availability to maize in the Tanzanian highlands. Different urea-N rates (0–150 kg N ha−1; split into two dressings) were applied to two soil types (TZi, sandy Alfisols; and TZm, clayey Andisols). In the early growing season, soil mineralized N was exposed to the leaching risk due to small crop N demand. In the second N application (major N supply accounting for two-thirds of the total N), applied urea was more efficient in increasing soil inorganic N availability at TZm than at TZi. Such effect of soil type could be the main contributor to the higher yield at TZm (up to 4.4 Mg ha−1) than that at TZi (up to 2.6 Mg ha−1) under the same N rate. The best-fitted linear-plateau model indicated that the soil inorganic N availability (0–0.3 m) at the tasseling stage largely accounted for the final yield. Further, yields at TZi were still limited by N availability at the tasseling stage due to fast depletion of applied-N, whereas yields plateaued at TZm once N availability was above 67 kg N ha−1. Our results provided a valuable reference for designing the N management to increase yield, while minimizing the potentially adverse losses of N to the environment, in different agro-ecological zones in SSA.