Origin of the Higher Difficulty in the Recognition of Vowels Compared to Handwritten Digits in Deep Neural Networks

Abstract

We investigate the origin of the significantly different error rates between handwritten digit machine recognition and vowel sound machine recognition. While the error rate for five Korean vowel sounds, [ɑ], [ʊ], [ɪ], [ο], and [ɛ], is about 10 percent, that of handwritten digit recognition is less than 1 percent for convolutional neural networks (CNNs) with raw data. We first dilute the information of the sound by subtracting its temporal fine structure, with the assumption that sorting out extraneous sound information will improve the accuracy of vowel recognition. Simulation results show no improvement though, indicating that the recognition rate difference does not arise from unnecessary sound information. Rather, conserving subtle information with no information reduction can be helpful to improve recognition rates; however, even the model with the highest accuracy does not reach the accuracy for handwritten digit recognition we desired. Finally, we find that the main difficulty of Korean vowel sound recognition comes from the similarity of [ο] and [ɛ]; without [ɛ], recognition of the remaining vowels is up to 99 percent. The similarity can be seen through their formant structure. Humans overcome the similarity to adeptly differentiate the two, and human vowel recognition remains far superior to the best performing CNNs. This indicates room to develop deep neural networks beyond the CNN still exists.