A novel vehicle collision detection system: Integrating audio-visual fusion for enhanced performance

Abstract

In vehicular accidents, the swift and accurate identification of car crashes is paramount, as it serves as the linchpin for prompt responses from emergency services and search-and-rescue operations. This study introduces an innovative multimodal car crash detection system that capitalizes on audio-visual data sourced from dashboard cameras, thus significantly enhancing the precision of automobile collision detection. In contrast to car crash detection systems relying on single-modal inputs, this research employs advanced multimodal methodologies with a neural network architecture that integrates Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) for car crash detection. This system concurrently leverages audio spectrogram and audio feature data, video RGB, video optical flow, and video differential information, enabling collision recognition through a neural network system, resulting in markedly improved collision detection accuracy. A comprehensive comparative analysis is conducted, benchmarking our system against the most cutting-edge existing car crash detection systems. The system employs a unique fusion approach, allowing it to apply its inference to audio-visual information of varying durations compared to the training set. The empirical findings unequivocally affirm our proposed solution’s substantial enhancement in car crash detection.