The differential subgrade settlement is a key issue in the operation of the high-speed railway, which affects the mechanical properties and service status of the track structure. To estimate the effects of the lateral differential settlement of the subgrade on the China Railway Track System (CRTS) II slab track, a 3-D finite element model was developed considering the damage behaviors of the concrete material and the interface cohesion of the slab track. The track deformation, the concrete damage, the interface damage were analyzed, respectively. The results indicate that the track structure has an obvious lateral tilt in the subgrade settlement section. The track structure on the side with the larger settlement is in a more unfavorable state. The concrete base and the precast slab are successively damaged when the settlement amplitudes of the subgrade are 20 mm and 40 mm respectively under the subgrade settlement condition with a settlement wavelength of 20 m. There exists a particular wavelength that maximizes the interface damage variable. The track structure is in a suspended state under the settlement condition with a large settlement amplitude and a short settlement wavelength.