Serial Weakening of Human-Based Attributes Regarding Their Effect on Content-Based Speech Recognition

Abstract

Numerous studies have investigated automatic speech recognition tasks, such as content-based speech recognition, using machine learning techniques, such as deep learning. In general, each speech sample contains four main human-based attributes: i.e., content, emotion, gender, and speaker identity. Among them, the content has the lowest correlation with the other three attributes. However, to classify speech samples concerning each attribute, the model ignores the existence of unrelated attributes. This study shows that information on these non-content attributes is not always useful and can cause a content-based speech classifier to significantly underperform. Moreover, weakening the effects of one, two, or three attributes is possible, and weakening these attributes in a specific order is crucial. For this purpose, two-input, two-output autoencoders are proposed as a feature extraction method. These networks are specifically designed to reduce the level of information (in this case, one, two, or three attributes). The level of change in the performance of classifiers caused by using these pre-trained autoencoders helps rank the negative effect of selected human-based attributes. Based on the results obtained, gender has the most negative effect on the performance of content-based speech recognition models, and serial weakening gives the best results when considering the attributes in the following order: gender, speaker identity, and emotion.