The cable bracket is one of the crucial components of high-speed Electric Multiple Units (EMUs), which is fixed to the axle box by bolts to support the cable. As an important parameter of the joint surface, contact stiffness significantly affects the dynamic response and service life of the cable bracket. However, it is difficult to directly measure the contact stiffness of the joint surface. As a result, an identification method is proposed in this investigation to identify the contact stiffness. A finite element model incorporating both normal and tangential stiffness parameters is developed. According to the finite element model and modal testing results, an improved particle swarm optimization algorithm is employed to identify the contact stiffness of the joint surface. The effect of the contact stiffness on the vibrations of the cable bracket is investigated. The findings indicate that pre-tightening torque significantly influences the contact stiffness of the cable bracket. The contact stiffness has a great effect on the vibrations of the cable bracket. Optimal contact stiffness notably reduces vibrations of the cable bracket, thereby extending its service life.