Speaker Change Research Articles

Comparative Analysis of Audio Features for Unsupervised Speaker Change Detection

This study examines how ten different audio features, including MFCC, mel-spectrogram, chroma, and spectral contrast etc., influence speaker change detection (SCD) performance. The analysis is conducted using two unsupervised methods: Bayesian information criterion with Gaussian mixture model (BIC-GMM), a model-based approach, and Kullback-Leibler divergence with Gaussian Mixture Model (KL-GMM), a metric-based approach. Evaluation involved statistical analysis of feature changes in relation to speaker changes (vice versa), supported by comprehensive experimental validation. Experimental results show MFCC as the most effective feature, demonstrating consistently good performance across both methods. Features such as zero crossing rate, chroma, and spectral contrast also showed notable effectiveness within the BIC-GMM framework, while mel-spectrogram consistently ranked as the least influential feature in both approaches. Further analysis revealed that BIC-GMM exhibits greater stability in managing variations in feature performance, whereas KL-GMM is more sensitive to threshold optimization. Nevertheless, KL-GMM achieved competitive results when paired with specific features, such as MFCC and zero crossing rate. These findings offer valuable insights into the impact of feature selection on unsupervised SCD, providing guidance for the development of more robust and accurate algorithms for practical applications.

Optimized technique for speaker changes detection in multispeaker audio recording using pyknogram and efficient distance metric.

Segmentation process is very popular in Speech recognition, word count, speaker indexing and speaker diarization process. This paper describes the speaker segmentation system which detects the speaker change point in an audio recording of multi speakers with the help of feature extraction and proposed distance metric algorithms. In this new approach, pre-processing of audio stream includes noise reduction, speech compression by using discrete wavelet transform (Daubechies wavelet 'db40' at level 2) and framing. It is followed by two feature extraction algorithms pyknogram and nonlinear energy operator (NEO). Finally, the extracted features of each frame are used to detect speaker change point which is accomplished by applying dissimilarity measures to find the distance between two frames. To realize it, a sliding window is moved across the whole data stream to find the highest peak which corresponds to the speaker change point. The distance metrics incorporated are standard "Bayesian Information Criteria (BIC)", "Kullback Leibler Divergence (KLD)", "T-test" and proposed algorithm to detect the speaker boundaries. At the end, threshold value is applied and their results are evaluated with Recall, Precision and F-measure. Best result of 99.34% is shown by proposed distance metric with pyknogram as compare to BIC, KLD and T-test algorithms.

Grounding with Structure: Exploring Design Variations of Grounded Human-AI Collaboration in a Natural Language Interface

Selecting an effective utterance among countless possibilities that match a user's intention poses a challenge when using natural language interfaces. To address the challenge, we leveraged the principle of least collaborative effort in communication grounding theory and designed three grounded conversational interactions: 1) a grounding interface allows users to start with a provisional input and then invite a conversational agent to complete their input, 2) a multiple grounding interface presents multiple inputs for the user to select from, and 3) a structured grounding interface guides users to write inputs in a structure best understood by the system. We compared our three grounding interfaces to an ungrounded control interface in a crowdsourced study (N=80) using a natural language system that generates small programs. We found that the grounding interfaces reduced cognitive load and improved task performance. The structured grounding interface further reduced speaker change costs and improved technology acceptance, without sacrificing the perception of control. We discuss the implications of designing grounded conversational interactions in natural language systems.

Neural Evidence for Syntactic Unification in Second Language Sentence Comprehension: A Time‐Frequency Analysis

AbstractThis study investigates whether syntactic unification occurs during online L2 sentence comprehension using time‐frequency analysis. We measured the oscillatory power changes in native English speakers and L1‐Cantonese L2‐English speakers while they were reading well‐formed English sentences, syntactically intact nonsense sentences, and random word lists. Additionally, we conducted traditional ERP analyses to test L2 speakers’ sensitivity to NP‐internal number (dis)agreement. The results show that low‐beta power significantly increased in the L2 group when reading not only well‐formed sentences but also nonsense sentences, replicating the pattern found in the L1 group. This suggests that syntactic unification occurs in L2 comprehension as reliably as in L1 comprehension. However, L2 speakers did not show increased positivity for NP‐internal number disagreement, indicating that they have not developed native‐like sensitivity to this syntactic error. The implications of these time‐frequency and ERP data for L2 sentence processing and syntactic development are discussed.

Tracking age-related changes in voice and speech production with Landmark-based analysis of speech.

Voice and speech production change with age, which can lead to potential communication challenges. This study explored the use of Landmark-based analysis of speech (LMBAS), a knowledge-based speech analysis algorithm based on Stevens' Landmark Theory, to describe age-related changes in adult speakers. The speech samples analyzed were sourced from the University of Florida Aging Voice Database, which included recordings of 16 sentences from the Speech Perception in Noise test of Bilger, Rzcezkowski, Nuetzel, and Rabinowitz [J. Acoust. Soc. Am. 65, S98-S98 (1979)] and Bilger, Nuetzel, Rabinowitz, and Rzeczkowski [J. Speech. Lang. Hear. Res. 27, 32-84 (1984)]. These sentences were read in quiet environments by 50 young, 50 middle-aged, and 50 older American English speakers, with an equal distribution of sexes. Acoustic landmarks, specifically, glottal, bursts, and syllabicity landmarks, were extracted using SpeechMark®, MATLAB Toolbox version 1.1.2. The results showed significant age effect on glottal and burst landmarks. Furthermore, the sex effect was significant for burst and syllabicity landmarks. While the results of LMBAS suggest its potential in detecting age-related changes in speech, increase in syllabicity landmarks with age was unexpected. This finding may suggest the need for further refinement and adjustment of this analytical approach.

Spoken Language Change Detection Inspired by Speaker Change Detection

Spoken language change detection (LCD) refers to identifying the language transitions in a code-switched utterance. Similarly, identifying the speaker transitions in a multispeaker utterance is known as speaker change detection (SCD). Since tasks-wise both are similar, the architecture/framework developed for the SCD task may be suitable for the LCD task. Hence, the aim of the present work is to develop LCD systems inspired by SCD. Initially, both LCD and SCD are performed by humans. The study suggests humans require (a) a larger duration around the change point and (b) language-specific prior exposure, for performing LCD as compared to SCD. The larger duration requirement is incorporated by increasing the analysis window length of the unsupervised distance-based approach. This leads to a relative performance improvement of 29.1%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$29.1\\%$$\\end{document} and 2.4%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$2.4\\%$$\\end{document}, and a priori language knowledge provides a relative improvement of 31.63%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$31.63\\%$$\\end{document} and 4.01%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$4.01\\%$$\\end{document} on the synthetic and practical codeswitched datasets, respectively. The performance difference between the practical and synthetic datasets is mostly due to differences in the distribution of the monolingual segment duration.

The Role of the Proxemic Factor in the Implementation of the Cooperative Strategy of Communication in the English Fairy Tale Discourse

The functional and communicative potential of proxemic components is investigated in this article on the basis of the English fairy tale discourse. The interaction of proxemic components and verbal communicative components is taken into consideration. The regularities of the influence of proxemic means on the fairy tale speakers’ achievement of communicative goals within the cooperative communicative strategy are established, and the discursive personality’s tactical toolkit for its implementation is identified. The modulation of distance between fairy tale speakers and the change of a fairy tale speaker’s posture in relation to his/her communicative partner are the most communicatively significant and effective proxemic means used in the process of communication. Insufficient diversity in the repertoire of proxemic components in comparison with other nonverbal components is their characteristic feature. As a result, proxemic components are frequently used in combination with kinesic and prosodic communicative components, accompanying the verbal components in the utterances of fairy tale speakers. In this way the strengthening of the semantic load of a fairy tale speaker’s utterance takes place. The proxemic components are also used by a fairy tale discursive personality for more effective implementation of various communicative tactics within the frames of the cooperative communicative strategy in the English fairy tale discourse.

Speech recognition based on mobile sensor networks application in English education intelligent assisted learning system

With the popularity of English learning and the development of mobile technology, the development of intelligent assisted learning system has become an effective learning method. The purpose of this study is to improve the speech recognition performance of English education intelligent learning assistance system by using mobile sensor networks, and improve the learning effect of learners. This paper uses a speech recognition algorithm based on mobile sensor network to acquire learners' speech signals with multiple sensor nodes. By preprocessing and feature extraction of acquired signals, machine learning algorithm is applied to recognize them, thus improving the recognition accuracy. The experimental results show that compared with traditional speech recognition methods, the speech recognition system based on mobile sensor network has better performance in the case of background noise and speaker change. When learners use the system to learn English, they can get more accurate and reliable speech recognition results, improve the learning effect and interactive experience, and provide a more effective learning tool for English learners.

Speech comprehension in the context of speaker changes: The importance of voice-feature continuity at the cocktail party

In this talk, I will show how different groups of listeners use voice acoustics to enhance speech comprehension under adverse listening conditions, specifically when the auditory scene comprises a multitude of sounds heard at once. These “cocktail-party”-like situations pose a difficult conceptual problem: To comprehend target speech, listeners need to attend to the target voice while at the same time ignoring other irrelevant sounds. The cocktail-party problem is made considerably easier when all target sounds are spoken by the same talker. Previous work suggests that such benefits from voice continuity can be—in large part—attributed to two prominent voice features: Glottal-pulse rate (GPR) and vocal-tract length (VTL). GPR determines the fundamental frequency of a speech sound and is perceived as vocal pitch; VTL determines the spectral envelope of a speech sound and is perceived as an aspect of vocal timbre. Apart from being important voice identity cues, GPR and VTL have been shown to play a crucial role in cocktail-party listening. Here, I will present data from a series of experiments highlighting the importance of voice-feature continuity for speech comprehension at the cocktail party.

Compensation for coarticulation despite a midway speaker change: Reassessing effects and implications.

Accounts of speech perception disagree on how listeners demonstrate perceptual constancy despite considerable variation in the speech signal due to speakers' coarticulation. According to the spectral contrast account, listeners' compensation for coarticulation (CfC) results from listeners perceiving the target-segment frequencies differently depending on the contrastive effects exerted by the preceding sound's frequencies. In this study, we reexamine a notable finding that listeners apparently demonstrate perceptual adjustments to coarticulation even when the identity of the speaker (i.e., the "source") changes midway between speech segments. We evaluated these apparent across-talker CfC effects on the rationale that such adjustments to coarticulation would likely be maladaptive for perceiving speech in multi-talker settings. In addition, we evaluated whether such cross-talker adaptations, if detected, were modulated by prior experience. We did so by manipulating the exposure phase of three groups of listeners by (a) merely exposing them to our stimuli (b) explicitly alerting them to talker change or (c) implicitly alerting them to this change. All groups then completed identical test blocks in which we assessed their CfC patterns in within- and across-talker conditions. Our results uniformly demonstrated that, while all three groups showed robust CfC shifts in the within-talker conditions, no such shifts were detected in the across-talker condition. Our results call into question a speaker-neutral explanation for CfC. Broadly, this demonstrates the need to carefully examine the perceptual demands placed on listeners in constrained experimental tasks and to evaluate whether the accounts that derive from such settings scale up to the demands of real-world listening.

Exploring Innovative Teaching Methods for the Integration of Interactive Digital Media and Traditional Drawing Techniques

Abstract Digital media is an important technological symbol of the new era, presenting significant technological support in the development of various industries. This project demonstrates an innovative method of teaching painting by integrating interactive digital media. Two art classes at a university are chosen to conduct experiments. In order to explore the quality of painting classroom interaction after integrating interactive digital media, a speech-based classroom speaking behavior measurement method is designed by using the effective speech detection algorithm based on Bi-LSTM, the speaker change point detection algorithm based on Bi-LSTM, and the speaker recognition algorithm based on TDNN and AP clustering. After the teaching experiment, comparing the drawing works, learning self-discipline performance and learning satisfaction of students in two classes, it was found that students who were taught drawing using interactive digital media were 6.4%, 10.94% and 14.44% higher in the corresponding dimensions. Meanwhile, in the analysis of classroom speaking behavior, the ratio of teacher and student speech fluctuated frequently, presenting a good quality of classroom interaction. The drawing teaching method integrating interactive digital media can enhance students’ drawing level and learning satisfaction, promote classroom atmosphere, and promote teacher-student interaction in drawing teaching.

Claiming insufficient knowledge in pairwork and groupwork classroom activities

This conversation analytic paper explores how students in pair and groupwork tasks produce and respond to claims of insufficient knowledge (CIKs). Based on 7 h of video recordings of peer interaction from 18 English as a Foreign Language (EFL) classes in Czech secondary schools, we analyze how students express and negotiate their epistemic status using CIKs: when producing a CIK, the current speaker assumes a not-knowing status, thus making the imminent speaker change more relevant, as the next speaker then typically reveals his or her epistemic status. We also show that when a CIK is produced dyadic interactions in second position, the first speaker then produces a knowledge display response, or another CIK, resulting in abandoning the question, which differs from sequences that can be found in frontal teaching. The findings also show that CIKs can be used to resume task-related talk and initiate repair sequences focusing on language issues that the task comprises. Thus, CIKs can be viewed as central interactional resources for students to manage the task, i.e., to invite others to contribute, to resume their talk, or to abandon the current question, and to initiate repair sequences focusing on problematic items from the task.

Communication between Astronauts and Nasa Deep Space Network: A Conversation Analysis

In the field of conversation analysis, conversation is one way to approach social order. Conversations are not the seemingly disorganized and random objects they seem at first. This is a special locus of interaction where changes in speakers are repeated (occur), the parties speak at the same time. Gapless and non-overlapping transitions are common. The aim of this study is to analyze aspects of conversation analysis use in communication between Astronauts and NASA Deep Space Network (DSN) under the framework. This study use qualitative descriptive method is applied in this research. The data are collected by transcribe their utterances from Youtube and transcribing their utterances directly into written text. The result of this study shows that 6 types of adjacency pairs, from the data analysis found there are 47 utterances that consist of adjacency pairs. From the findings, there are 7 requesting – agreement, 11 assessment – agreement, 17 question – answer, 1 compliment – acceptance, 1 complaint – apology, 10 offer - acceptance for adjacency pairs.

The use of online interactive teaching mode in university music teaching under the background of informationization

Abstract This research method uses vocal recognition, classroom interaction analysis and social network analysis theory to cluster audio data through the process of data collection and feature extraction, active speech detection and speaker change detection, and finally quantifies and visualizes classroom data based on S-T analysis and social network analysis methods, and calculates corresponding indicators for qualitative analysis. After reforming the teaching mode, 80% of students expressed great satisfaction. The information-based teaching tools resulted in an overall upward trend in the grades of the three classes. In the middle and high score range of 80-89, Class 1 increased by 34%, Class 2 by 37%, and Class 3 by 28%, and all three classes showed a more significant increase. Regarding classroom interaction behaviors, teachers' verbal behavior rates decreased from 55.86% and 44.01% to 27.48%, while students' verbal behavior rates increased from 21.92% and 21.23% to 39.73%.

Transition-Relevance Places Machine Learning-Based Detection in Dialogue Interactions

A transition-relevance place (TRP) represents a place in a conversation where a change of speaker can occur. The appearance and use of these points in the dialogue ensures a correct and smooth alternation between the speakers. In the presented article, we focused on the study of prosodic speech parameters in the Slovak language, and we tried to experimentally verify the potential of these parameters to detect TRP. To study turn-taking issues in dyadic conversations, the Slovak dialogue corpus was collected and annotated. TRP places were identified by the human annotator in the manual labelling process. The data were then divided into chunks that reflect the length of the interpausal dialogue units and the prosodic features were computed. In the Matlab environment, we compared different types of classifiers based on machine learning in the role of an automatic TRP detector based on pitch and intensity parameters. The achieved results indicate that prosodic parameters can be useful in detecting TRP after splitting the dialogue into interpausal units. The designed approach can serve as a tool for automatic conversational analysis or can be used to label large databases for training predictive models, which can help machines to enhance human-machine spoken dialogue applications.

Longitudinal fNIRS and EEG metrics of habituation and novelty detection are correlated in 1–18-month-old infants

IntroductionHabituation and novelty detection are two fundamental and widely studied neurocognitive processes. Whilst neural responses to repetitive and novel sensory input have been well-documented across a range of neuroimaging modalities, it is not yet fully understood how well these different modalities are able to describe consistent neural response patterns. This is particularly true for infants and young children, as different assessment modalities might show differential sensitivity to underlying neural processes across age. Thus far, many neurodevelopmental studies are limited in either sample size, longitudinal scope or breadth of measures employed, impeding investigations of how well common developmental trends can be captured via different methods. MethodThis study assessed habituation and novelty detection in N = 204 infants using EEG and fNIRS measured in two separate paradigms, but within the same study visit, at 1, 5 and 18 months of age in an infant cohort in rural Gambia. EEG was acquired during an auditory oddball paradigm during which infants were presented with Frequent, Infrequent and Trial Unique sounds. In the fNIRS paradigm, infants were familiarised to a sentence of infant-directed speech, novelty detection was assessed via a change in speaker. Indices for habituation and novelty detection were extracted for both EEG and NIRS ResultsWe found evidence for weak to medium positive correlations between responses on the fNIRS and the EEG paradigms for indices of both habituation and novelty detection at most age points. Habituation indices correlated across modalities at 1 month and 5 months but not 18 months of age, and novelty responses were significantly correlated at 5 months and 18 months, but not at 1 month. Infants who showed robust habituation responses also showed robust novelty responses across both assessment modalities. DiscussionThis study is the first to examine concurrent correlations across two neuroimaging modalities across several longitudinal age points. Examining habituation and novelty detection, we show that despite the use of two different testing modalities, stimuli and timescale, it is possible to extract common neural metrics across a wide age range in infants. We suggest that these positive correlations might be strongest at times of greatest developmental change.

Top-down effect of dialogue coherence on perceived speaker identity

A key mechanism in the comprehension of conversation is the ability for listeners to recognize who is speaking and when a speaker switch occurs. Some authors suggest that speaker change detection is accomplished through bottom-up mechanisms in which listeners draw on changes in the acoustic features of the auditory signal. Other accounts propose that speaker change detection involves drawing on top-down linguistic representations to identify who is speaking. The present study investigates these hypotheses experimentally by manipulating the pragmatic coherence of conversational utterances. In experiment 1, participants listened to pairs of utterances and had to indicate whether they heard the same or different speakers. Even though all utterances were spoken by the same speaker, our results show that when two segments of conversation are spoken by the same speaker but make sense for different speakers to say, listeners report hearing different speakers. In experiment 2 we removed pragmatic information from the same stimuli by scrambling word order while leaving acoustic information intact. In contrast to experiment 1, results from the second experiment indicate no difference between our experimental conditions. We interpret these results as a top-down effect of pragmatic expectations: knowledge of conversational structure at least partially determines a listener’s perception of speaker changes in conversation.

Modeling of an Automatic Vision Mixer With Human Characteristics for Multi-Camera Theater Recordings

Laymen are not trained in camera control and how to use a vision mixing desk to switch between different cameras in a video production. Specific training and expert knowledge are required. In the non-professional environment, multi-camera recordings of theater performances or other stage performances are therefore difficult to realize. This can be remedied by a production process with high-resolution cameras from which image sections of long shots or medium long shots are cropped manually or automatically in post-production. One approach to automate the generation of suitable image cuts could be to focus on speaker changes so that the person who is speaking is shown in the generated cut. However, these image cuts can appear static and robotic if they are set too precisely. Therefore, the characteristics and habits of professional vision mixers (persons who operate the vision mixing desk) during the editing process are investigated in more detail in order to incorporate them into an automation process. The characteristic features of five different vision mixers are examined, which were used under almost identical recording conditions for theatrical cuts in TV productions. The cuts are examined with regard to their temporal position in relation to pauses in speech, which take place during speaker changes on stage. It is shown that different professional vision mixers set the cuts individually differently before, in or after the pauses in speech. From the analysis of the image cuts, an approach for a model is developed in which the individual characteristics of a vision mixer can be set. With the help of this model, a more human appearance can be given to otherwise exact and robotic cuts, when automating image cuts.

Analyzing Noise Robustness of Cochleogram and Mel Spectrogram Features in Deep Learning Based Speaker Recognition

The performance of speaker recognition systems is very well on the datasets without noise and mismatch. However, the performance gets degraded with the environmental noises, channel variation, physical and behavioral changes in speaker. The types of Speaker related feature play crucial role in improving the performance of speaker recognition systems. Gammatone Frequency Cepstral Coefficient (GFCC) features has been widely used to develop robust speaker recognition systems with the conventional machine learning, it achieved better performance compared to Mel Frequency Cepstral Coefficient (MFCC) features in the noisy condition. Recently, deep learning models showed better performance in the speaker recognition compared to conventional machine learning. Most of the previous deep learning-based speaker recognition models has used Mel Spectrogram and similar inputs rather than a handcrafted features like MFCC and GFCC features. However, the performance of the Mel Spectrogram features gets degraded in the high noise ratio and mismatch in the utterances. Similar to Mel Spectrogram, Cochleogram is another important feature for deep learning speaker recognition models. Like GFCC features, Cochleogram represents utterances in Equal Rectangular Band (ERB) scale which is important in noisy condition. However, none of the studies have conducted analysis for noise robustness of Cochleogram and Mel Spectrogram in speaker recognition. In addition, only limited studies have used Cochleogram to develop speech-based models in noisy and mismatch condition using deep learning. In this study, analysis of noise robustness of Cochleogram and Mel Spectrogram features in speaker recognition using deep learning model is conducted at the Signal to Noise Ratio (SNR) level from −5 dB to 20 dB. Experiments are conducted on the VoxCeleb1 and Noise added VoxCeleb1 dataset by using basic 2DCNN, ResNet-50, VGG-16, ECAPA-TDNN and TitaNet Models architectures. The Speaker identification and verification performance of both Cochleogram and Mel Spectrogram is evaluated. The results show that Cochleogram have better performance than Mel Spectrogram in both speaker identification and verification at the noisy and mismatch condition.

Polymorphism of second person singular forms of address in the Spanish of Medellin, Colombia

This study examines polymorphism of second person singular forms of address (2PS) in the Spanish of Medellin, Colombia. Polymorphism of 2PS is the usage of more than one 2PS directed toward the same person during the same interaction. Speakers from Medellin (N = 72) carried out an oral discourse completion task that consisted of 24 situations. Results showed that polymorphism is common in Medellin Spanish as it occurred 22.7% of the time. It was also found that polymorphism is constrained by three variables: change in speech act, speaker sex, and interlocutor relationship. In particular, a change in speech act, male speakers, and intimate interlocutors favored polymorphism. In addition, characteristics of polymorphism are identified. Findings demonstrate that polymorphism is not random but rather systematic and purposeful—it allows speakers to express different speech acts and emotions via the shift in 2PS. Results are discussed in light of broader implications and the future of polymorphism in the Spanish of Medellin.