The long-term excessive use of chemical fertilizers may result in soil degradation, but manure and straw application is considered to be an effective approach for alleviating this problem. The aim of this study is to examine the long-term impacts of different fertilization patterns on soil quality variables in a wheat–sweet potato rotation system. Four treatments were conducted in a field trial for a duration of twelve years, including (1) no fertilizer (control, CK); (2) application of mineral fertilizers (NPK) alone; (3) NPK with crop straw return (NPKs); (4) combined use of NPK and farmyard manure (NPKm). Thirteen physical, chemical, and biological soil parameters were measured. The results showed that the NPKm and NPKs significantly improved the proportion of macroaggregates (>0.25 mm) by 24.7% and 21.9% compared to the NPK alone, respectively. The proportion of microaggregates (0.053–0.25 mm) under the NPKm was 47.4% significantly higher than the NPKs. Additionally, the NPKm resulted in a 22.2% and 19.6% increase in the SOC content than the NPK and NPKs, respectively. In terms of soil-available K, the NPKs resulted in levels that were 42.1% and 49.6% higher than the NPKm and NPK alone, respectively. Long-term fertilization significantly decreased soil pH by 0.95–1.85 units compared to the control, whereas manure application could alleviate soil acidification, as shown when the pH increased by 10.6–18.7%. The NPKm and NPKs resulted in significantly increased soil pHs by 10.6% and 18.7% compared to the NPK alone, respectively. In addition, the NPKm and NPKs increased N-acetyl-β-D-glucosaminidase activity by 52.6% and 60.3% compared to the NPK alone. Determined by the minimum data set method, the NPKm treatment exhibited the highest soil quality index, followed by the NPKs and NPK. Our findings suggested that the combined use of chemical fertilizers with organic amendments proved beneficial for enhancing soil quality.