Abstract
Oral evaluation is one of the most critical processes in children’s language learning. Traditionally, the Scoring Rubric is widely used in oral evaluation for providing a ranking score by assessing word accuracy, phoneme accuracy, fluency, and accent position of a tester. In recent years, by the emerging demands of the market, oral evaluation requires not only providing a single score from pronunciation but also in-depth, meaning comments based on content, context, logic, and understanding. However, the Scoring Rubric requires massive human work (oral evaluation experts) to provide such deep meaning comments. It is considered uneconomical and inefficient in the current market. Therefore, this paper proposes an automated expert comment generation approach for oral evaluation. The approach first extracts the oral features from the children’s audio as well as the text features from the corresponding expert comments. Then, a Gated Recurrent Unit (GRU) is applied to encode the oral features into the model. Afterwards, a Long Short-Term Memory (LSTM) model is applied to train the mappings between oral features and text features and generate expert comments for the new coming oral audio. Finally, a Generative Adversarial Network (GAN) is combined to improve the quality of the generated comments. It generates pseudo-comments to train the discriminator to recognize the human-like comments. The proposed approach is evaluated in a real-world audio dataset (children oral audio) collected by our collaborative company. The proposed approach is also integrated into a commercial application to generate expert comments for children’s oral evaluation. The experimental results and the lessons learned from real-world applications show that the proposed approach is effective for providing meaningful comments for oral evaluation.
Highlights
Oral evaluation is a language-testing process, which includes pronunciation accuracy, fluency, integrity, logical ability, understanding ability and so on
With the development of deep learning, researchers have proposed a large number of acoustic model (AM) methods based on deep neural networks in speech recognition, which is generally divided into hybrid acoustic models and end-to-end acoustic models
Generative Adversarial Network (GAN)-Based Neural Audio Caption Model is composed of two neural networks, a generative neural network and a discriminative neural network
Summary
Oral evaluation is a language-testing process, which includes pronunciation accuracy, fluency, integrity, logical ability, understanding ability and so on. Our previous work had tried to apply the caption generation model to generate expert comment for the oral evaluation [11]. A Neural Audio Caption Model (NACM) is proposed to generate expert comments from the oral audio. Compared with the previous work, GNACM can produce more accurate and complete expert comment for the oral evaluation. We propose a model called NACM that can generate expert comment for the oral audio. Based on NACM, we propose an improved model called GNACM that can generate more accurate and complete expert comment for the oral audio. Beyond the Scoring Rubric approach, the work is the early try to generate expert comments for the oral evaluation
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have