Martensitic stainless steel containing 13% Cr–4% Ni suffers cavitation erosion (CE) as the common material of hydro turbine impellers. Two 13% Cr–4% Ni stainless steel samples were obtained by different melting and heating processes. One was of relatively low toughness but high ductility (LTHD), and the other was of relatively high toughness but low ductility (HTLD). This paper is to clarify the relationship between the mechanical properties and the CE resistance of the experimental steel samples. The CE of the two materials was studied using an ultrasonic vibration cavitation erosion rig. Mass loss, morphological observation, nanoindentation characterization, and tensile tests were employed to clarify the erosion mechanism. The results showed that LTHD stainless steel had slightly higher ductility, but lower toughness than HTLD material. The mass loss method verified that the CE resistance of LTHD material was higher than that for the HTLD material. In addition, both materials had an incubation stage of 2 h in the distilled water. The SEM revealed that material removal was preferentially initiated from the grain boundaries and slip zone after the incubation period. The ductility could delay the fracture of the material, which contributed more to cavitation erosion resistance than the toughness of the materials. The hardness test showed few relationships with the CE resistance.