An improved mesh stiffness calculation model of helical gear pair is proposed, in which the transverse gear tooth stiffness, transverse gear foundation stiffness, in addition to axial gear tooth stiffness, axial gear foundation stiffness, that influenced by surface roughness under elastohydrodynamic lubrication (EHL) condition are fully considered. The time varying friction coefficients are obtained by elastoplastic model under varying surface roughness, and then the influence of friction force on mesh stiffness calculation are investigated. The results show that the improved mesh stiffness calculation model could obtain the mesh stiffness under actual lubrication condition compared with conventional method, in which the axial mesh forces and friction forces acting on the gear tooth and gear foundation are always ignored in the mesh stiffness calculation for helical gear pair.