The deterioration of the elastic pads in the fastening system in high-speed turnout areas increases longitudinal stiffness irregularities along the track, posing potential hazards to wheel-rail impact and train running safety. This paper takes the frog area of No. 18 turnout as an example and constructs a vehicle-turnout rigid-flexible coupling model considering the detailed structure of the fastener elastic pads. By increasing the stiffness of the rubber pad under the tie plate to simulate the deterioration of the elastic pads in the turnout area, the study investigates the effects of deterioration location, degree, and number on the dynamic characteristics of the vehicle-turnout system, providing theoretical guidance for stiffness optimization and maintenance of the turnout area. The results indicate that: 1) the deterioration of the elastic pads significantly affects wheel-rail interaction and safety indicators, while having a minor effect on vehicle running stability; 2) the 95th sleeper (point rail with a top width of 60 mm) is identified as the most critical location for the deterioration of elastic pads; 3) comprehensively considering, the stiffness replacement limit for elastic pads in ballastless turnout areas can be set to twice the design stiffness, which is 50 kN/mm.