In this study, the Hoek Brown criteria is primarily employed to calculate the distribution of excavation damaged zone (EDZ) around the cavity buried at 500 m, 1000 m, and 2000 m based on the variation of the in-situ stress and the mechanical properties of the rockmass. Then, the simulation scheme Spectral Element Method (SEM) is applied to investigate the dynamic responses of the excavation-damaged cavity that subjected to the dynamic disturbance, including peak particle velocity (PPV), particle vibration frequency (PVF), particle displacement, and dynamic stress concentration factors (DSCF). Three two-dimensional (2D) models embedding a circular hole with different shapes of EDZ are prepared to numerically study the dynamic responses of the underground cavity, finding the influencing factors, such as the shapes of EDZ, the incident direction of the dynamic disturbance, and the dynamic source frequency. The cavity that is surrounded by EDZ has resembled as the one with a series of low-velocity areas. The results indicate that the underground cavity and EDZ become a scatter to the incoming stress waves that correspondingly produce high amplitude scattered waves. The influencing factors to the dynamic response of the cavity are summarized to be the shape of EDZ, the angle of stress disturbances, and the wavelength of incoming stress waves. We also find that the amplitude of stress disturbance around the cavity is proportional to the areas of EDZ and the frequency of the incident waves. In particular, only the tensile stress concentrates around the cavity that is surrounded by EDZ.