For studying the traffic safety of the high-speed railway, this research considers high-quality second-level wind speed data as its basis. However, the quality of second-level wind speed data can be greatly lowered by disturbances during data collection and storage. Therefore, it is crucial to control the data quality during collection and storage. Wind speed data along the high-speed railway are unstable and non-linear. In order to adapt to this characteristic, this study combines a convolutional neural network (CNN), long short-term memory (LSTM), and isolated forest from the time dimension to form a quality control (QC) algorithm for wind speed monitoring data. First, CNN is used to extract the original data features, which are then transferred to the LSTM network for one-step prediction. The prediction residual of the model is obtained and sent to the isolated forest, where the abnormal value position in the original wind speed data is calibrated by detecting the abnormal value position in the prediction residual. Comparative experiments have been conducted to test the performances of the three different QC methods. The results show that the error detection rate of CNN–LSTM–IF in this research method is approximately 0.95. For different terrains and seasons, the method has certain robustness and generalization.