Challenges of Vehicle Classification Using Acoustics

Abstract

Automated acoustic classification of vehicles is a challenging problem with many variables. Vehicles produce complex sounds from many sources, sound signatures vary between similar vehicles, and background noise has a large impact on audio data. Past research studied different vehicle classification techniques but often relied on datasets with little variation in vehicle model, environmental conditions, or microphones. High-accuracy results on these datasets suggest issues of overfitting. This paper highlights the challenges of creating robust datasets for difficult acoustic classification tasks. Challenges include collecting on multiple instances of a target class, recording with varied environmental noise, and collecting across multiple types of microphones. The work presented also evaluates classification performance on combinations of acoustic feature sets and common machine learning algorithms. Classification F-score performance drops 38.67% when the test set has background noise that differs from the classifier’s training set. Performance also significantly drops when the classifier is tested on a vehicle of a model year not included in the training set. Lastly, the vehicle classifier heavily overfits on the signatures of the microphones it is trained on.