High nitrogen (N) loading may contribute to recession of submerged macrophytes in shallow lakes; yet, its influences vary depending on environmental conditions. In August 2013, we conducted a 28-day factorial-designed field mesocosm experiment in Lake Taihu at the Taihu Laboratory for Lake Ecosystem Research (TLLER) to examine the effects of high N loading on the growth of Vallisneria natans in systems with contrasting sediment types. We ran the experiments with two levels of nutrient loading—present-day external nutrient loading (average P: 5 μg·L−1·day−1, N: 130 μg·L−1·day−1) and P: 5 μg·L−1·day−1, and with three times higher N loading (N: 390 μg·L−1·day−1) and used sediment with two contrasting nutrient levels. V. natans growth decreased significantly with increasing N loading, the effect being dependent, however, on the nutrient status of the sediment. In low nutrient sediment, relative growth rates, leaf biomass and root biomass decreased by 11.9%, 18.2% and 23.3%, respectively, at high rather than low N loading, while the decline was larger (44.0%, 32.7% and 41.8%, respectively) when using high nutrient sediment. The larger effect in the nutrient-rich sediment may reflect an observed higher shading of phytoplankton and excess nutrient accumulation in plant tissue, though potential toxic effects of the high-nutrient sediment may also have contributed. Our study confirms the occurrence of a negative effect of increasing N loading on submerged plant growth in shallow nutrient-enriched lakes and further shows that this effect is augmented when the plants grow in nutrient-rich sediment. External N control may, therefore, help to protect or restore submerged macrophytes, especially when the sediment is enriched with nutrients and organic matter.