Winter road surface condition classification using convolutional neural network (CNN): visible light and thermal image fusion

Abstract

During winter, road conditions play a crucial role in traffic flow efficiency and road safety. Icy, snowy, slushy, and wet road conditions reduce tire friction and affect vehicle stability which could lead to dangerous crashes. To keep traffic operations safe, cities spend a significant budget on winter maintenance operations such as snow plowing and spreading of salt and sand. This paper proposes a methodology for automated winter road surface condition classification using convolutional neural network (CNN) and the combination of thermal and visible light cameras. As part of this research, 4244 pairs of visible light and thermal images are captured from pavement surfaces and classified into snowy, icy, wet, and slushy surface conditions. Two single-stream CNN models (visible light and thermal streams) and one dual-stream CNN model are developed. The average F1-Score of dual-stream model is 0.866, 0.935, 0.985, and 0.888 on snowy, icy, wet, and slushy, respectively. The weighted average F1-Score is 0.94.