Quenched-partitioning-tempered (Q-P-T) steel, with a combination of high strength and high toughness, is a potential wear-resistant material. However, there is disagreement that Q-P-T steels could replace conventional wear-resistant materials due to its unclear degradation mechanism during sliding wear. To promote the application of Q-P-T steel, the relationship between its microstructure obtained at different partitioning temperatures, microstructural damage behavior during sliding wear, and wear resistance was investigated. The results showed that the fatigue spalling was the main cause of the volume loss because of the formation of deformed layer consisting of equiaxed nanocrystals in the worn subsurface. Moreover, as the fatigue cracks nucleated and propagated at the interface of deformed layer/matrix, improving the matrix’s impact toughness can retard the formation of fatigue crack. The Charpy impact energy of Q-P-T steel first increased and then decreased with the elevation of partitioning temperature for the transformation of residual austenite morphology, achieving a maximum of 41 J at 280 °C. Consequently, the wear volume decreased and then increased with increasing partitioning temperature, showing a minimum value at 280 °C.