This paper numerically investigates the validity of Rayleigh damping model considering explicit operator to assess the dynamic response of high rise buildings under seismic loads. Considering transverse and longitudinal seismic waves, the bending moment, shear force, axial force, and and inter story drift are evaluated for a Core wall and a frame system of 46 story each. It is found that considering the explicit scheme, the dynamic responses are amplified significantly especially for axial forces. The reported amplification can be attributed to the ignorance of stiffness proportional Rayleigh damping coefficient which is associated with the stability issue of the implemented explicit operator. The paper indicates that Rayleigh damping model does not provide the required/expected damping for the higher modes of higher frequencies hence, it is not appropriate to be used along with the explicit operator especially for buildings of wide range of frequencies. It is worth pointing out that for classical dynamic analysis which follows the implicit scheme, Rayleigh damping seems to well consider the higher modes of high frequencies with higher damping ratio in comparison to the initial mode shapes. Consequently, the literature explicit scheme should be revised to accurately consider a proper damping for the higher modes which is crucial to assess the total response of structures of long periods and wide range of frequencies such as high rise buildings among others.