Plasma nitriding as a surface modification was applied on two substrate materials: cast iron D6510 and cast steel S0050A. After measurement of the friction coefficients of the treated samples using a pin-on-disc tribotester, an inclined impact-sliding wear tester was utilized to investigate their tribological behaviour under tilting contact with extremely high contact pressure. While numerous surface fatigue cracks, severe chipping, and peeling of the compound layer were observed for the treated cast steel sample, the treated cast iron sample had far fewer surface fatigue cracks without chipping or peeling of the compound at the same test condition. The governing mechanisms of the treated cast iron sample’s superior resistance to surface fatigue failure were revealed by studying the cross-sectional hardness and nitrogen concentration profile. Energy-dispersive X-ray spectroscopy (EDS) analysis indicated that the treated cast iron sample had a smaller nitrogen concentration gradient, which led to a smaller hardness gradient as measured. The results suggest that a smaller hardness gradient between the compound layer and the diffusion zone and a thicker hardened case was able to improve the wear resistance and surface fatigue cracking resistance against high contact loads. Moreover, the smaller friction coefficient of the treated cast iron sample could also be beneficial for improving the wear resistance.