Metallic materials exhibit significant size effect at mesoscopic scale have been revealed by many experiments. Thus, it is essential to consider size effect when conducting mesoscopic scale studies in tribology. However, there is no constitutive model considering both ductile fracture and size effects of rough surface in the literature. At the same time, the simulation of friction and wear at the mesoscopic scale suffer from computational inefficiency using the conventional theory of mechanism-based strain gradient (CMSG) plasticity theory. To address the aforementioned challenges, we propose a mesoscopic coupled plasticity-damage model by combining the coupled plasticity-damage model and the simplified CMSG plasticity theory. This model can take into account not only the effect of stress state, temperature, strain rate on plasticity and fracture, but also the effect of the effective plastic strain gradient. Meanwhile, the model is able to solve the effective plastic strain gradient by a simplified method, thereby enhancing the computational efficiency. Then, model parameters for bearing bushing material of an engine are calibrated. Finally, scratch simulation and scratch tests of bearing bushing material at the mesoscopic scale are conducted to validate the effectiveness of the mesoscopic constitutive model. The residual scratch depth and width, coefficient of friction at different normal loads are investigated by experiment and simulation, respectively. Results proved the effectiveness of mesoscopic coupled plasticity-damage model, which is applicable for investigating issues related to wear, friction, and contact.