Two-stream video-based deep learning model for crashes and near-crashes

Abstract

The use of videos for effective crash and near-crash prediction can significantly enhance the development of safety countermeasures and emergency response. This paper presents a two-stream hybrid model with temporal and spatial streams for crash and near-crash identification based on front-view video driving data. The novel temporal stream integrates optical flow and TimeSFormer, utilizing divided-space–time attention. The spatial stream employs TimeSFormer with space attention to complement spatial information that is not captured by the optical flow. An XGBoost classifier merges the two streams through late fusion. The proposed approach utilizes data from the Second Strategic Highway Research Program Naturalistic Driving Study, which encompasses 1922 crashes, 6960 near-crashes, and 8611 normal driving segments. The results demonstrate excellent performance, achieving an overall accuracy of 0.894. The F1 scores for crashes, near-crashes, and normal driving segments were 0.760, 0.892, and 0.923, respectively, indicating strong predictive power for all three categories. The proposed approach offers a highly effective and scalable solution for identifying crashes and near-crashes using front-view video driving data and has broad applications in the field of traffic safety.