The increase in the speed of high-speed trains and the widespread use of lightweight vehicle body technology have made vehicle systems more sensitive to aerodynamic excitation. However, few studies have focused on the worsening of passenger ride comfort, as opposed to just safety, when trains pass along windy railway lines. This study first analyzes the connection between flow structure and aerodynamic load, using bidimensional empirical mode decomposition (BEMD). Then, it examines the effect of aerodynamic loads on the vibration characteristics of vehicles and human bodies, at different speeds under crosswinds, by combining a vehicle-track-seat-human body coupling model. Finally, the effects of crosswinds on vehicle and human body vibration are analyzed, including differences in vibration characteristics of the human head when in different positions. The BEMD results indicate that the flow field around the vehicle is dominated by low-frequency oscillation. These low-frequency components show close alignment with the aerodynamic loads’ main frequency, and the high-speed trains’ natural frequency, which may lead to vehicle resonance shaking, and to derailment. Additionally, it is found that lateral vibration comfort for human legs and thighs is worst in both the head and middle cars, while vertical vibration comfort for the human head is worst in the head car. When running at three speeds under crosswinds, fluctuations in aerodynamic loads and track irregularities are major causes of discomfort for humans, particularly affecting vertical vibration comfort. In crosswinds, vertical vibration of the human head is significantly greater than lateral vibration, indicating a loss of comfort, primarily in the vertical direction, when trains are excited by crosswinds. It is observed that lateral vibration comfort is worst at the front end of the head car, but best at their rear end. Vertical vibration comfort is best in the middle section of the head car.