Aeolian sand hazards are often a threat to culverts, which are important channels and pieces of infrastructure of the desert railway. In addition to wind speed, wind direction, and culvert structure, terrain may also be an important reason for the formation of culvert sand hazards. However, there are few studies on the effect of terrain on the sediment accumulation characteristics of culverts. This paper established computational fluid dynamics (CFD) models of railway culverts (flat and concave culverts) based on Euler’s two-fluid theory. An analysis of the influence of terrain on the distribution law of the flow fields and sand accumulation around railway culverts was carried out. The results show that the horizontal wind speed curves changes in a “W” shape along the centre axis surface from the forecourt to the rearcourt within a range of 30 m~66.8 m. Low-speed backflow is formed at the inlet and outlet of the culvert, and the minimum wind speed reaches −3.6 m/s and −4.2 m/s, respectively, when the height from the bottom of the culvert is 1.0 m and 1.5 m, resulting in intensified sand sedimentation. In concave culverts, the lower the roadbed height, the easier it is for sand to accumulate at the culvert outlet, the rearcourt, and the track; the sand volume fraction is close to 0.63, affecting the normal operation of the trains. On the contrary, the higher the roadbed, the easier it is for sand to accumulate at the culvert inlet, hindering the passage of engineering vehicles and reducing the function of the culverts. These results reveal that terrain plays a pivotal role in the sand accumulation around culverts and that it should be one of the key considerations for the design of new railway culverts. This work can provide a theoretical basis for preventing and managing sand hazards in railway culverts.