Railway-induced vibrations in building structures require predictions to assess the serviceability. Fundamental properties such as the predominant modes of railway-induced vibrations are critical in structural design and mitigation approaches. However, the existing numerical models are customised to specific buildings. This renders these models unsuitable for broader mechanistic investigations. To address this, this study proposes a rod-sprung-mass (RSM) model to investigate the dynamic behaviour of structures under vertical ground excitations induced by railways. The columns were modelled as axial vibration rods, with each floor suspended on the rod as a sprung mass. The model was validated using a three-story steel frame structure and a five-story concrete structure. Subsequently, the dynamic characteristics of the structures and railway-induced responses were discussed. The structural vertical modes are attributed to the interaction between the columns and slabs, and the structural natural frequencies are outside the frequency range of the individual components. Parametric studies revealed that lower-order structural modes are generally predominant. Under such circumstances, the floor response tends to first decrease and subsequently increase with the floor height. As a result, low-rise buildings exhibit the maximum amplitude on the top floor, whereas the maximum amplitude is on the ground floor for high-rise buildings. Meanwhile, when structural high-order modes become predominant owing to a high-frequency input, the responses of the middle floors intensify, and the peak responses can be altered to the middle floors. This study emphasises the necessity of considering vertical global modes, whereas existing studies have focused mainly on mode shapes on particular floors. The RSM model is an efficient tool for structural design and vibration control.