Information on the long-term effects of the addition of organics and fertilizers to wheat under the pearl millet–wheat cropping system with semi-arid conditions in north-western India is still lacking. The present research was conducted in an ongoing field experiment initiated during Rabi 1995 at the Research Farm of Chaudhary Charan Singh at Haryana Agricultural University, Hisar. After 25 years, the impacts of nutrient management practices on soil fertility and wheat productivity were evaluated. The experiment comprised a total of eight treatment combinations viz. half and full doses of recommended fertilizers (N and P), organic manures (FYM: farmyard manure, POM: poultry manure, and PRM: press mud) alone and in combination with NP fertilizers. The conjoint application of organic manure and chemical fertilizers resulted in a positive influx of nutrients via increasing total organic carbon (TOC), available N, P, K, and S, which ranged from 0.46 to 1.42%, 122.70 to 194.70, 15.66 to 74.92, 340.5 to 761.2, and 15.26 to 54.63 kg ha−1 in surface soil (0–15 cm), respectively. Carbon fractions and crop yield were significantly improved by adopting integrated nutrient management (INM). The TOC showed a positive and significant correlation with C fractions (r > 0.92) and with soil-available N, P, K, and S (r > 0.77) content. The data also revealed a strong relationship between TOC and soil-available (0–15 cm) nutrients i.e., available N (R2= 0.769), available P (R2 = 0.881), available K (R2 = 0.758), and available S (R2 = 0.914), respectively. Thus, practices that increased TOC were also beneficial in enhancing the availability of the nutrients in the soil. A positive and highly significant correlation was also found among wheat yield, nutrient (NPKS) content, and uptake. A polynomial relationship between grain yield and grain N (R2 = 0.962), P (R2 = 0.946), and K (R2 = 0.967) content, and between straw yield and straw N (R2 = 0.830), P (R2 = 0.541) and K (R2 = 0.976) content was obtained. Integrated use of PRM7.5 followed by FYM15 and POM5 coupled with NP fertilizers proved best, which could be beneficial for obtaining nutritious and highest wheat yield (grain: 6.01 t ha−1 and straw: 7.70 t ha−1) coupled with improved fertility within a sustained manner under the pearl millet–wheat sequence in prevailing semi-arid conditions of the North Indian state of Haryana.