Neural network prediction of sound quality via domain Knowledge-Based data augmentation and Bayesian approach with small data sets

Abstract

This study proposes a novel deep learning methodology to evaluate the interior noise in vehicles on mechanical and affective levels by employing small data sets. A convolutional neural network (CNN) model is constructed from the frequency-rpm spectrograms of vehicle noises to predict the mechanical attributes of the noise. The noises are classified based on the number of engine cylinders (3, 4, 6, and 8). Owing to the high variability in spectrograms, mathematical expressions for the engine order lines are derived to augment the training data. With respect to the affective attributes, three classes (i.e., ‘sporty,’ ‘powerful,’ and ‘luxurious’) are selected for noise characterization. The spectrograms are used to design another CNN-based classification model that predicts the affective attributes from the perspective of experts. Although expert knowledge is employed for data labeling, a quarter of the data remains unlabeled, owing to inherent subjectivity. The model is trained on the labeled data and is validated by comparing the predicted class probabilities for the unlabeled data and their distribution in the RGB color space. K-fold cross-validation is used to evaluate the reliability of the model. A regression model is built based on Bayesian inference to evaluate the affective attributes of the noise from the perspective of end users. Given four singular values from the matrix of sound quality metrics, the model predicts the mean jury ratings for noise. The classification models exhibit generalization performances of 98.2% and 91.6%, respectively, and the regression model exhibits a mean squared error of 2.57×10-3, thereby demonstrating the applicability of the proposed approach to vehicle noise analysis.