ABSTRACTThe scuffing load capacity of gear is closely related to the meshing temperature rise of tooth surface. The key to predict the temperature rise is to establish an accurate meshing temperature rise model. In the paper, a tribo‐dynamic model of helical gear is established through coupling of tooth surface lubrication parameters, and the influence of temperature rise on ambient temperature during meshing process is considered. Then, the effects of oil supply temperature, input speed and torque on tooth surface temperature rise, film thickness, friction excitation and gear dynamic characteristics are discussed. The results show that the temperature rise of the gear is higher during the engaging‐in and engaging‐out regions. Meanwhile, there is local high temperature at the end of the contact line due to the end effect. The vibration of gear along the off‐line‐of‐action direction is mainly determined by friction excitation. With the increase of oil supply temperature, input speed and torque, the risk of scuffing failure increases and the influence of oil supply temperature and input load is more significant. The conclusions of this paper may provide some valuable suggestions for the anti‐gluing failure design of gear in engineering.