Concerning the cumulative damage of building structures caused by the long-term effects from high-speed railway train loads and various environmental impacts, and to ensure the service safety of building structures around viaducts, this paper establishes a vibration safety assessment method for the entire life cycle of building structures with less workload and good applicability, while ensuring the accuracy and effectiveness of the prediction of vibration response of building structures. Firstly, the vibration source function is formed based on the mechanical model of the mobile vibration source of high-speed railway trains, and the vibration wave of high-speed railway trains is numerically simulated. Then the time and frequency characteristics of the simulated high-speed railway train vibration waves are verified through on-site monitoring tests. In addition, the peak particle vibration velocity attenuation formula for the foundation site of the viaduct is derived by the dimension analysis method, and it is used to correct the attenuation characteristics of the simulated high-speed railway train vibration waves. Finally, based on the response spectrum method to realize the prediction and assessment of the overall vibration response of the building structures, and using the cumulative damage coefficient to reflect the cumulative degree of the service damage of the building structure, a vibration safety assessment method for the whole lifecycle of the building structure is proposed. The method overcomes the shortcoming that vibration velocity does not reflect the impact of service damage on the vibration safety of building structures, and avoids neglecting localized vibration damage, and is therefore useful for the rapid, overall vibration safety assessment of any long-term service building structure in a viaduct site.