Under long-term tram loads, cracking occurs on the composite pavement surrounding an embedded tram track. However, in the current pavement design of the shared right-of-way section of a tram track, the trackside pavement fatigue characteristics are not considered. The investigation of the real dynamic strain response of the trackside composite pavement under tram loads is crucial for pavement system design. In this paper, a three-dimensional finite element model of tram-track- composite pavement was established. The rationality of this model was verified by a field pavement dynamic test. The influence of the track irregularity and tram speed on the dynamic characteristics of the pavement surrounding the track was explored. The trackside pavement strain distribution along the longitudinal and vertical directions of the track had been studied. The high-frequency strain components caused by the random shortwave track irregularity excitation were found in the strain frequency domain curves in different directions, and the track irregularity amplifies the influence of the tram speed on the surrounding pavement dynamic response. The loading frequency is an important input for determining the dynamic model and fatigue characteristics of asphalt materials. At the same time, the lab test loading frequency and fatigue life corresponding to the mixture were calculated by the mechanistic coupled model and empirical damage evolution equation. It provides an analytical base for the design of pavement surrounding embedded tram tracks.