Trust-worth multi-representation learning for audio classification with uncertainty estimation

Abstract

Multi-view learning has been explored for audio classification tasks, exploiting different representations of audio signals, ranging from MFCC, CQT, to raw signals. The quality of each view may vary for different audio signals, and the appropriate uncertainty quantification for each view has not been fully explored. In this work, we explore a trusted multi-view learning framework for classification tasks in order to fully incorporate different views. Our framework consists of three parallel branches of Transformer architectures (Gammatone spectrogram, log-Mel and CQT) and they are combined using the uncertainty estimation of different branch. In addition to computing the classification probabilities, the uncertainty of each representation can also be obtained using the framework. We firstly calculate the evidence based on feature vectors to obtain the probabilities and the uncertainty of classification problems for Gammatone, log-Mel and CQT branch. By integrating the confidence from each of the different representations using the Dempster–Shafer theory, the classification framework can provide higher accuracy and confidence. To demonstrate the effectiveness of the proposed framework, we conduct the experiments on the GTZAN dataset. The obtained results show that our method can reach the accuracy of 83.0%, which significantly outperforms single representation-based methods while providing uncertainty estimation for different views.