Although phosphorus (P) fertilizer application is widely used to improve production, irrational P application has a negative impact on the zinc (Zn) nutrition of cereal crops. Previous researchers observed and confirmed that P application decreases grain Zn concentrations and bioavailability in cereal crops. However, it remains unclear whether different P fertilizer types can alleviate the antagonism of P and Zn in the soil and grain and, thus, enhance the Zn nutritional level of cereal crops while maintaining production. Thus, a completely randomized pot experiment was conducted on winter wheat grown in two calcareous soils (lime concretion black soil and fluvo-aquic soil). Five P fertilizer types (single superphosphate, diammonium phosphate, fused calcium–magnesium phosphate, triple superphosphate, and ammonium polyphosphate, abbreviated, respectively, as SSP, DAP, FMP, TSP, and APP) were applied to each soil compared to no P fertilizer (CK). Plant and topsoil samples were collected during the flowering and maturity stages of winter wheat, and biomass, Zn concentrations in each organ, and grain phytic acid concentrations were analyzed. Grain yield was not affected by the application of different P fertilizer types to lime concretion black soil, while it was significantly increased by the application of TSP and APP to fluvo-aquic soil. The application of DAP and APP effectively promoted soil available Zn concentrations in both calcareous soils. In lime concretion black soil, the application of FMP significantly increased Zn remobilization to grains, while the application of DAP increased post-anthesis Zn uptake, thereby increasing grain Zn concentrations and its bioavailability. In fluvo-aquic soil, post-anthesis Zn remobilization and uptake were significantly increased by the application of TSP and APP, finally achieving higher grain Zn concentrations and Zn harvest index and effectively promoting grain Zn bioavailability. In conclusion, the rational application of DAP to wheat grown in lime concretion black soil and of TSP or APP to fluvo-aquic soil can achieve superior grain Zn nutrition quality while concurrently retaining high production and high P use efficiency, reducing micronutrient deficiency and further contributing to green agricultural development and human health.