Text and Sound-Based Feature Extraction and Speech Emotion Classification for Korean

Abstract

Embracing the complexities of human emotions conveyed through speech, this study ventures into Speech Emotion Recognition (SER) within the human-computer interaction domain, leveraging cutting-edge artificial intelligence technologies. Focusing on the auditory attributes of speech, such as tone, pitch, and rhythm, the research introduces an innovative approach that amalgamates deep learning techniques with the A Learnable Frontend for Audio Classification (LEAF) algorithm and wav2vec 2.0 pre-trained on a large corpus, specifically targeting Korean voice samples. This methodology underlines the capacity of these technologies to process and decipher complex vocal expressions, aiming to elevate emotion classification precision notably. The exploration extends the horizons of SER by accentuating auditory emotion cues and aspires to enrich machine interactions to be more intuitive and empathetic across various applications like healthcare and customer service. The outcomes underscore the efficacy of transformer-based models, particularly wav2vec 2.0 and LEAF, in capturing the subtle emotional states expressed in speech, thereby affirming the importance of auditory cues over conventional visual and textual indicators. The study's implications for further research herald a promising trajectory for evolving AI systems adept at nuanced emotion detection, thereby forging pathways toward more natural and human-centric interactions between individuals and machines. This advancement is crucial for developing empathetic AI that can seamlessly integrate into our daily lives, understanding and reacting to human emotions in a way that mirrors human understanding and compassion.