Real-time detection of the mechanical state of ballast bed during the tamping operation in railway maintenance is of great significance for improving the effectiveness of operations. In this study, a novel test method named the track shifting test was proposed based on the track realigning operation of the tamping vehicle. The track panel was pushed by the shifting device. Moreover, the lateral resistance of ballast bed was reflected through easily measured indexes. An accurate coupling model of the shifting device and the ballasted track was constructed. Based on the model, the mechanical response of ballast and the track panel induced by the shifting load was analyzed. Results indicated that at an effective loading displacement of 2mm, the lateral resistance of ballast bed within a detectable range of up to five sleepers can be inverted by the shifting force and the displacement of sleepers. A machine learning model was established to obtain the mapping relationship between the shifting force, the displacement of sleepers, and the lateral resistance of ballast bed. Therefore, real-time detection of the lateral resistance was achieved by combining the proposed test method and the machine learning algorithm. This study can contribute to the synchronous detection of the mechanical state of ballast bed during tamping operation.