Pronunciation Assessment Research Articles

AI 기반 영어 교육용 프로그램과 한국인 교사의 초등학생 영어 발음평가 비교 연구

AI 기반 영어 교육용 프로그램과 한국인 교사의 초등학생 영어 발음평가 비교 연구

Pronunciation assessment of learners, by learners, and for learners: Effects, validity and reliability, and learners’ perception

Studies have reported comparable effectiveness of teacher assessment and learner assessment in promoting second-language (L2) writing skills (Vuogan & Li, 2022; Yu & Lee, 2016). However, whether learner assessment is equally helpful in enhancing L2 pronunciation learning remains underexplored. Addressing this gap, this pretest–posttest quasi-experimental study examined the effects, validity and reliability, and learners’ perception of peer assessment and self-assessment in L2 English pronunciation. The participants, 183 English majors at a university in China, were divided into three groups, each receiving one type of assessment for the segmental and suprasegmental modules of a pronunciation course. The participants’ performances were examined by pre- and post-intervention reading tasks and assessment forms, and their perceptions by a questionnaire survey. Repeated-measures ANOVA tests of the reading task results revealed similar performances by the three groups on segmentals and suprasegmentals. Cronbach’s alpha tests of scores in the assessment forms across the weeks showed high validity and reliability for peer assessment but marginal validity and acceptable reliability for self-assessment. Repeated-measures one-way ANOVA tests and a paired-samples t-test of the questionnaire survey results revealed high confidence in evaluating others’ and own pronunciation but a general preference for teacher assessment, which was perceived to be more effective than the other two types. These findings suggest that L2 English learners can be more self-reliant in pronunciation learning and should be encouraged and guided to be so.

A Critique of the Computer-Based English Speaking Test in Fujian (CEST-FJ)

This essay applies Bachman and Palmer’s (1996) Test Usefulness Framework to scrutinize the reliability, construct validity, and impact of the Computer-based English Speaking Test in Fujian (CEST-FJ). The CEST-FJ’s inclusion in the National College Entrance Examination (NCEE) represents a positive step towards promoting communicative ability in English language learning in China. The computerized administration and scoring processes enhance the test’s reliability and control over construct-irrelevant variance. However, limitations such as potential variations in difficulty, construct under-representation of communicative competence, challenges for pronunciation assessment in the context of English as an International Language (EIL), and the non-compulsory and low-value nature of the test have been identified. Additionally, the lack of transparency surrounding the CEST-FJ hampers assessment and accountability. This essay emphasizes the need for more theoretical and empirical research to justify or enhance the test in the future, particularly in light of its high-stakes nature and influence on test-takers’ university admissions. Test designers are urged to prioritize transparency and provide more information to facilitate further research and ensure the effectiveness of the CEST-FJ.

Digital enhancement of pronunciation assessment: Automated speech recognition and human raters*

Digital enhancement of pronunciation assessment: Automated speech recognition and human raters*

Sztuczna inteligencja jako narzędzie do przełamywania barier językowych w mówieniu po angielsku

The aim of this article is to present artificial intelligence tools that help overcome language barriers and enable English language learners to communicate effectively in English. The author of the article shows the results of a study by the international education company EF Education First regarding the level of English proficiency among adults from different countries. He then explains why so many people learning English have problems with speaking, discusses different language barriers, and the role of artificial intelligence in improving English speaking skills. The author analyzes the impact of chatbots on developing speaking skills in English and provides examples of tools which use artificial intelligence to improve speaking abilities, including pronunciation assessment.

Microsoft Reading Progress as Capt Tool

The paper explores the accuracy of feedback provided to non-native learners of English by a pronunciation module included in Microsoft Reading Progress. We compared pronunciation assessment offered by Reading Progress against two university pronunciation teachers. Recordings from students of English who aim for native-like pronunciation were assessed independently by Reading Progress and the human raters. The output was standardized as negative binary feedback assigned to orthographic words, which matches the Microsoft format. Our results indicate that Reading Progress is not yet ready to be used as a CAPT tool. Inter-rater reliability analysis showed a moderate level of agreement for all raters and a good level of agreement upon eliminating feedback from Reading Progress. Meanwhile, the qualitative analysis revealed certain problems, notably false positives, i.e., words pronounced within the boundaries of academic pronunciation standards, but still marked as incorrect by the digital rater. We recommend that EFL teachers and researchers approach the current version of Reading Progress with caution, especially as regards automated feedback. However, its design may still be useful for manual feedback. Given Microsoft declarations that Reading Progress would be developed to include more accents, it has the potential to evolve into a fully-functional CAPT tool for EFL pedagogy and research.

The Effects of the SQ4R in Developing Grade 6 Students’ Pronunciation of Consonant Clusters

The ability to pronounce consonant clusters is essential for the communicative processes of a language. The SQ4R teaching model, normally employed in reading comprehension development studies, could be adapted to benefit a speaking classroom. The purposes of the study were 1) to investigate the effects of the SQ4R technique on Thai students’ consonant cluster pronunciation and 2) to study student satisfaction with the SQ4R technique in developing consonant cluster pronunciation. The participants were 40 grade 6 students in a public school in Thailand. The participants were chosen using the purposive sampling method. The criteria emphasize the heterogeneous characteristics of first language, experiences in foreign countries, and dialects. The instruments were 1) a learning management plan designed using the activities of the SQ4R technique in developing the pronunciation of consonant clusters, 2) consonant clusters pronunciation assessment form, and 3) a satisfaction questionnaire. The data were analyzed using percentages, mean scores, standard deviation, and an effective index with the determining criteria of 75/75. The results of the study indicate the benefits of the SQ4R technique to both in-process and end-product outcomes. Moreover, it was also found that the participants were satisfied with the processes provided by the learning management. The results of the study provide academic implications for further research and pedagogical implication for educational personnel seeking to solve problems in developing students’ pronunciation.

Implementation of a System for Assessing the Quality of Spoken English Pronunciation Based on Cognitive Heuristic Computing.

This paper analyzes and investigates the quality assessment of spoken English pronunciation using a cognitive heuristic computing approach and designs a corresponding spoken pronunciation quality assessment system for practical training. Using the general Goodness of Pronunciation assessment algorithm as a benchmark, the shortcomings of the traditional Goodness of Pronunciation method are explored through statistical experiments, and the validity of the overall posterior probability output from the speech model for pronunciation quality assessment is verified. For the analysis of rhythm, there is no common algorithm framework, but in this paper, the F0 similarity algorithm based on dynamic time regularization and the stop similarity algorithm based on forced alignment is proposed for the two main factors of rhythm, intonation, and pause, respectively. After framing, the Hamming window processing is used to make the signal smoother, reduce the side lobe size after fast Fourier transform processing, and solve the problem of spectrum leakage. Compared with the ordinary rectangular window function, the Hamming window can obtain a higher quality spectrum. And combined with CTC for speech recognition modeling, the recognition rates are comparable in the case of using BLSTM and bidirectional threshold cyclic unit BGRU as the hidden layer unit, respectively, and the training time is 23% less than BLSTM using BGRU; in addition, the BGRU-CTC model is improved by using a 2-BGRU-CTC model with 256 hidden layer nodes, so that the error rate of phoneme recognition is reduced to 33%. The effectiveness of the algorithm framework is also verified through experiments, which further proves the effectiveness of our proposed phoneme segment feature and rhyme similarity algorithm.

Алгоритм и методика количественной оценки схожести речевых сигналов

The paper proposes a method to solve the task of automated quantitative assessment of the syllable pronunciation. This quantitative assessment is used to evaluate the speech quality during speech rehabilitation. An algorithm for quantifying the similarity of two audio signals of different lengths is presented. The algorithm uses a hybrid match measure. The hybrid measure is based on calculation of three metrics (DTWdistance, correlation coefficient and Minkowski metric) and using a fuzzy classifier as a mechanism for combining the calculated values. The average number of coincidences of estimates by the proposed algorithm and estimates by the previously applied method is 83%. A method for quantifying the similarity of speech signals using several reference signals is proposed. The method allows to consider the variability of speech and the individual characteristics of the phoneme’s pronunciation. This is achieved by using records of the patient's preoperative speech as reference signals.

The Implementation of Intelligibility-Based Pronunciation Instruction on a Pre-Sessional Eap Course in the Uk

Pre-sessional EAP access courses for learners of English intending to study on undergraduate or postgraduate courses in the UK are faced with particular challenges in incorporating meaningful suprasegmental pronunciation instruction into their programmes. This research examines the pronunciation goals of teachers, course leaders, and learners on a ten week UK pre-sessional access course, particularly with regard to suprasegmental instruction, how these goals are reflected in pronunciation assessment, and how teacher goals are informed by their attitudes and beliefs. A mixed methods approach, including direct observation and semi-structured interviews, is employed to address the area of enquiry. Results derived from course documents and a semi-structured interview show a lack of clarity of course goals. Although there is a general emphasis on suprasegmental rather than segmental instruction, in semi-structured interviews teachers report a lack of course goals. Assessment and practice do not always adhere to a goal of intelligibility, and support for teachers, in terms of the materials and how they might be exploited seems limited.

Benefits of Adaptive Learning Transfer From Typing-Based Learning to Speech-Based Learning.

Memorising vocabulary is an important aspect of formal foreign-language learning. Advances in cognitive psychology have led to the development of adaptive learning systems that make vocabulary learning more efficient. One way these computer-based systems optimize learning is by measuring learning performance in real time to create optimal repetition schedules for individual learners. While such adaptive learning systems have been successfully applied to word learning using keyboard-based input, they have thus far seen little application in word learning where spoken instead of typed input is used. Here we present a framework for speech-based word learning using an adaptive model that was developed for and tested with typing-based word learning. We show that typing- and speech-based learning result in similar behavioral patterns that can be used to reliably estimate individual memory processes. We extend earlier findings demonstrating that a response-time based adaptive learning approach outperforms an accuracy-based, Leitner flashcard approach in learning efficiency (demonstrated by higher average accuracy and lower response times after a learning session). In short, we show that adaptive learning benefits transfer from typing-based learning, to speech based learning. Our work provides a basis for the development of language learning applications that use real-time pronunciation assessment software to score the accuracy of the learner’s pronunciations. We discuss the implications for our approach for the development of educationally relevant, adaptive speech-based learning applications.

English Lingua Franca: New Parameters for the Teaching (and Testing) of English Pronunciation?

The recent (2018) Companion Volume to the Common European Framework offers an overhaul of many of the scales of descriptors, including, notably, phonology. A single, skeletal, scale for ‘phonological control’ is replaced by three scales, describing overall control, sound articulation, and prosodic features. In each of these, the focus has become intelligibility, rather than proximity to a native speaker accent. In this article I examine the development of pronunciation teaching since the communicative revolution, and the rise of English as a lingua franca (ELF) in which intelligibility is crucial. The article concludes with a reflection on how (if at all) the revised framework could inform an ELF aware assessment of pronunciation.

Automatic Speech Recognition (ASR) Systems Applied to Pronunciation Assessment of L2 Spanish for Japanese Speakers

General-purpose automatic speech recognition (ASR) systems have improved in quality and are being used for pronunciation assessment. However, the assessment of isolated short utterances, such as words in minimal pairs for segmental approaches, remains an important challenge, even more so for non-native speakers. In this work, we compare the performance of our own tailored ASR system (kASR) with the one of Google ASR (gASR) for the assessment of Spanish minimal pair words produced by 33 native Japanese speakers in a computer-assisted pronunciation training (CAPT) scenario. Participants in a pre/post-test training experiment spanning four weeks were split into three groups: experimental, in-classroom, and placebo. The experimental group used the CAPT tool described in the paper, which we specially designed for autonomous pronunciation training. A statistically significant improvement for the experimental and in-classroom groups was revealed, and moderate correlation values between gASR and kASR results were obtained, in addition to strong correlations between the post-test scores of both ASR systems and the CAPT application scores found at the final stages of application use. These results suggest that both ASR alternatives are valid for assessing minimal pairs in CAPT tools, in the current configuration. Discussion on possible ways to improve our system and possibilities for future research are included.

Using Automatic Speech Recognition to Facilitate English Pronunciation Assessment and Learning in an EFL Context

With the advancement of automatic speech recognition (ASR) technology, ASR-based pronunciation assessment can diagnose learners' pronunciation problems. Meanwhile, ASR-based pronunciation training allows more opportunities for pronunciation practice. This study aims to investigate the effectiveness of ASR technology in diagnosing English pronunciation errors and to explore teachers' and learners' attitudes towards using ASR technology as a pronunciation assessment tool and as a learning tool. Five Chinese EFL learners participated in read-aloud tests, including a human-assessed test and an ASR-assessed test. Pronunciation error types diagnosed by the two tests were compared to determine the extent of overlapping areas. The findings demonstrate that there were overlaps between human rating and machine rating at the segmental level. Moreover, it was found that learners' varied pronunciation learning needs were met by using the ASR technology. Implications of the study will provide insights relevant to using ASR technology to facilitate English pronunciation assessment and learning.

What Characterizes Comprehensible and Native‐like Pronunciation Among English‐as‐a‐Second‐Language Speakers? Meta‐Analyses of Phonological, Rater, and Instructional Factors

AbstractThe current study presents two meta‐analyses to explore what underlies the assessment and teaching of comprehensible and nativelike pronunciation among English‐as‐a‐Second‐Language speakers. In Study 1, listener studies (n = 37) were retrieved examining the influence of segmental, prosodic, and temporal features on listeners’ intuitive judgements of comprehensibility and nativelikeness/accentedness as per different listener backgrounds (expert, mixed, L2). In Study 2, training studies (n = 17) were retrieved examining the effects of segmental, prosodic, and temporal‐based instruction on ESL learners’ pronunciation. The results showed that (a) comprehensibility judgements were related to a range of segmental, prosodic, and temporal features; (b) accentedness judgements were strongly tied to participants’ correct pronunciation of consonants and vowels; and (c) instruction led to larger gains in comprehensibility than in nativelikeness. Moderator analyses demonstrated that expert listeners were more reliant on phonological information. Greater effects of instruction on comprehensibility than nativelikeness became clearer, especially when the treatment targeted prosodic accuracy. The findings suggest that ESL practitioners should prioritize suprasegemental practice to help students achieve comprehensible L2 pronunciation. The attainment of nativelike pronunciation, by contrast, may require an exclusive focus on the refinement of segmental accuracy, which is resistant to the influence of instruction.

Speech Processing for Language Learning: A Practical Approach to Computer-Assisted Pronunciation Teaching

This article contributes to the discourse on how contemporary computer and information technology may help in improving foreign language learning not only by supporting better and more flexible workflow and digitizing study materials but also through creating completely new use cases made possible by technological improvements in signal processing algorithms. We discuss an approach and propose a holistic solution to teaching the phonological phenomena which are crucial for correct pronunciation, such as the phonemes; the energy and duration of syllables and pauses, which construct the phrasal rhythm; and the tone movement within an utterance, i.e., the phrasal intonation. The working prototype of StudyIntonation Computer-Assisted Pronunciation Training (CAPT) system is a tool for mobile devices, which offers a set of tasks based on a “listen and repeat” approach and gives the audio-visual feedback in real time. The present work summarizes the efforts taken to enrich the current version of this CAPT tool with two new functions: the phonetic transcription and rhythmic patterns of model and learner speech. Both are designed on a base of a third-party automatic speech recognition (ASR) library Kaldi, which was incorporated inside StudyIntonation signal processing software core. We also examine the scope of automatic speech recognition applicability within the CAPT system workflow and evaluate the Levenstein distance between the transcription made by human experts and that obtained automatically in our code. We developed an algorithm of rhythm reconstruction using acoustic and language ASR models. It is also shown that even having sufficiently correct production of phonemes, the learners do not produce a correct phrasal rhythm and intonation, and therefore, the joint training of sounds, rhythm and intonation within a single learning environment is beneficial. To mitigate the recording imperfections voice activity detection (VAD) is applied to all the speech records processed. The try-outs showed that StudyIntonation can create transcriptions and process rhythmic patterns, but some specific problems with connected speech transcription were detected. The learners feedback in the sense of pronunciation assessment was also updated and a conventional mechanism based on dynamic time warping (DTW) was combined with cross-recurrence quantification analysis (CRQA) approach, which resulted in a better discriminating ability. The CRQA metrics combined with those of DTW were shown to add to the accuracy of learner performance estimation. The major implications for computer-assisted English pronunciation teaching are discussed.

L2-ruotsin lausepainon sanallinen kuvaaminen – menetelmällisiä käänteitä ääntämisen arviointiin


 
 
 In this study, we explore how sentence stress is described in pronunciation assessment. The data, also used in two previous studies, consist of listener ratings of Finnish-speaking learners’ pronunciation in read-aloud and free speech. The listeners rated the speakers’ pronunciation on segmental and suprasegmental level on a numeric scale and verbally. Typical descriptions of sentence stress in the data focus on specific error categories and general comments. The sentence stresses that were assessed with the lowest ratings were often described in more detail than those with higher ratings. According to the raters’ comments, the sentence stresses that got the lowest ratings typically occurred too often or not often enough. The sentence stresses with the highest ratings were also described with positive comments. There were differences between the rater groups when it comes to the amount of comments and the proportion of specific and general comments. Differences between read-aloud and free speech concerned the proportion of general comments and the frequency of too many stressed words. The results can be utilized in pronunciation assessment, as assessing pronunciation is often considered challenging.
 
 

Pronunciation Problems of Indonesian EFL Learners in Pronouncing /g/ Sound

Pronunciation is essential in delivering speech. However, pronouncing English word is difficult since the students speak Indonesian as their native language. The common problems might be caused by the differences between sound systems of the languages, especially in /g/ sound. This study objective to find how well the students’ mastery in pronouncing /g/ sound and common problems faced by them. This study highlighting two research problems; (1) how well the fourth semester students do in pronouncing /g/ sound and (2) what problems are faced by them in pronouncing /g/ sound. It is a descriptive qualitative study using triangulation data involving twenty five students of fourth semester students of English Language Education Program Faculty of Cultural Studies Universitas Brawijaya. The instruments were adapted and adopted from Dewi (2009) and developed by the researcher. The instruments are pronunciation test, recorder, assessment rubric, and questionnaire. Data analysis were done statistically and descriptively. The result of this study were (1) mostly the students’ mastery in pronouncing /g/ sound in all positions are good (81.5%) with the following details; excellent (pronouncing silent /g/ before /n/ at the end of a word, silent /g/ before /n/ in the same syllable, soft /g/ sound); good (pronouncing silent /g/ before /n/ at the beginning of a word and hard /g/ sound); average (pronouncing the exception); and poor (pronouncing silent /g/ before /m/ in the same syllable and words with /gh/ sound), (2) problems faced by the fourth semester students are the differences of sound systems between their native language and their foreign language and students’ motivation in pronouncing /g/ sound correctly (such as checking pronunciation or phonetic transcription in dictionary).

Towards a speech therapy support system based on phonological processes early detection

Phonological disorders are characterized by substitutions, insertion and/or deletions of sounds during the process of language acquisition, which are known as Phonological Processes (PPs). In the speech therapy domain, an early identification of PPs allows the diagnosis and treatment of various pathologies and may improve clinical tasks, however, there are few proposals that focus on the identification of PPs for supporting Speech-Language Pathologists (SLPs). Recent research applied Case-Based Reasoning (CBR) in medical domain to identify specific cases related to patients. Situation-Awareness (SA) is a technique that allows computing systems to adapt itself and respond to users or other systems according to environment information. Moreover, there is no indicative in related literature of CBR and SA being used for detecting PPs that may occur in pronunciation. In this paper, we introduce the union of SA and CBR, tied to machine learning algorithms for proposing a system to predict PPs, supporting specialists in their clinical decisions. To evaluate the system, we implemented it in a software architecture prototype and evaluated the prototypes using a knowledge base containing near one hundred thousand audio files, collected from more than 1,000 pronunciation assessments. The evaluation of the prototypes showed an accuracy over 93% in the prediction of PPs, resulting in a efficient tool for clinical decision support and therapeutic planning. We also presented a direct qualitative comparison between our approach and related work.

MOBILE LEARNING IN DEVELOPING PHONETIC COMPETENCE OF FUTURE INTERPRETERS

The article considers teaching English phonetics to future interpreters with mobile learning in the light of the competence approach. The authors analyse the possibilities of applying mobile learning as an additional source and demonstrates the feasibility of its implementation in teaching English phonetics. The primary aim of this research is to examine the effectiveness of m-learning on the development of future interpreters’ English phonetic competence. Such activities used in mobile learning as “Group Mind Mapping of Key Phonetic Issues”, “Creating Event-Glide Calendar as a Guide of a Coursework”, “E-Presentations on Teaching and Learning Phonetics”, “Real-Time Testing”, “Public Pronunciation Assessment”, “Different Perception” are described and analysed. During the pedagogical experiment, the criteria (cognitive; pragmatic; reflective), indicators, and levels (high, average and low) of the formation of future interpreters’ English phonetic competence through mobile learning were determined. 50 students of the Faculty of English philology and methodology of teaching English at Bohdan Khmelnytsky Melitopol State Pedagogical University took part in the experiment. The results prove that applying mobile learning in teaching English phonetics provides an opportunity to increase the level of English phonetic competence of future interpreters as well as encourages them for self-development and creative work.