HAPOVER: A Haptic Pronunciation Improver Device

Abstract

This research article describes the HAPOVER (a haptic pronunciation improver device), which was produced to help second language learners' pronunciation skills as part of their foreign language learning process. The device comprises eight independent vibration motors connected to the fingers other than the thumb. The developed tactile feedback device was utilized to enhance pronunciation via speech processing and using vibration motor. The acoustic difference between the reference and the participant's recorded pronunciation signal was calculated with the help of Mel Frequency Cepstral Coefficients (MFCC) and Dynamic Time Warping (DTW) and converted into haptic codes that guide the speaker. The article describes the haptic codes' features and the creation method thereof. The device was tested by psychophysical evaluation. The results indicate an 80.74% improvement in word pronunciation with haptic guidance post training. Therefore, the results corroborate that word pronunciation data can be transmitted via tactile codes. © 2024 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.