Speech Domain Research Articles

Toward Domain Adaptive Learning-Based Variation Autoencoder Emotional Analysis in English Teaching

Speech emotion analysis plays an important role in English teaching by analyzing the reading state of students. Teachers can dynamically adjust the teaching content according to the emotional feedback of students and improve the teaching quality of the school. Due to unstable student emotions and background noise, the accuracy of speech emotion recognition is constrained. Although multimodal data can alleviate the deficiency of a single modality, collecting and annotating multimodal samples requires a significant amount of resources. To resolve this issue, this paper proposes a novel multimodal sentiment analysis framework based on domain adaptive learning mechanisms to assist English teaching. We construct a novel multi-task variation autoencoder framework in which we simultaneously complete reconstruction and classification tasks. To improve speech emotion recognition performance, we introduce domain adaptive learning based on the Wasserstein distance between two variational hidden layers from the video domain (source domain) and speech domain (target domain). To validate the effectiveness of our proposed model, we conducted extensive comparative experiments on two public datasets and a self-built English oral dataset. All experimental results indicate that domain adaptation learning mechanisms can effectively improve the recognition performance of the target domain. On the self-built dataset for English teaching, the proposed model achieves higher performance compared to other deep models.

The Sensitivity and Specificity of the Korean-Infant and Toddler Checklist in Screening for Developmental Language Delay in Infants and Toddlers

Objectives: This study aimed to investigate the validity of the Korean-infant and toddler checklist (K-ITC) in screening for developmental language delay through analysis of sensitivity and specificity.Methods: Participants were 39 infants with developmental language delay aged 6-24 months, and 39 typically developing infants matched for chronological age. Total scores and scores from three domains (social, speech, and symbolic) of the K-ITC were compared between the two groups using independent samples t-tests. Receiver operating characteristic (ROC) analysis was conducted to assess the sensitivity and specificity of each score, confirming the validityResults: Infants with developmental language delay scored significantly lower than typically developing infants on the social and speech domains of the K-ITC, but not on the symbolic domain. ROC analysis indicated significant area under the curve (AUC) values for total scores and scores from each domain, with sensitivity of 95% for total scores and 87.18% for domain scores, and specificity of 64.10% for total scores. Sensitivity was notably higher compared to specificityConclusion: This study confirms that the K-ITC exhibits high sensitivity as a screening tool for developmental language delay in infants, demonstrating validity in significantly identifying developmental delays in infants and toddlers.

HiddenSinger: High-quality singing voice synthesis via neural audio codec and latent diffusion models

Recently, denoising diffusion models have demonstrated remarkable performance among generative models in various domains. However, in the speech domain, there are limitations in complexity and controllability to apply diffusion models for time-varying audio synthesis. Particularly, a singing voice synthesis (SVS) task, which has begun to emerge as a practical application in the game and entertainment industries, requires high-dimensional samples with long-term acoustic features. To alleviate the challenges posed by model complexity in the SVS task, we propose HiddenSinger, a high-quality SVS system using a neural audio codec and latent diffusion models. To ensure high-fidelity audio, we introduce an audio autoencoder that can encode audio into an audio codec as a compressed representation and reconstruct the high-fidelity audio from the low-dimensional compressed latent vector. Subsequently, we use the latent diffusion models to sample a latent representation from a musical score. In addition, our proposed model is extended to an unsupervised singing voice learning framework, HiddenSinger-U, to train the model using an unlabeled singing voice dataset. Experimental results demonstrate that our model outperforms previous models regarding audio quality. Furthermore, the HiddenSinger-U can synthesize high-quality singing voices of speakers trained solely on unlabeled data.

Phoneme Recognition in Korean Singing Voices Using Self-Supervised English Speech Representations

In general, it is difficult to obtain a huge, labeled dataset for deep learning-based phoneme recognition in singing voices. Studying singing voices also offers inherent challenges, compared to speech, because of the distinct variations in pitch, duration, and intensity. This paper proposes a detouring method to overcome this insufficient dataset, and applies it to the recognition of Korean phonemes in singing voices. The method started with pre-training the HuBERT, a self-supervised speech representation model, on a large-scale English corpus. The model was then adapted to the Korean speech domain with a relatively small-scale Korean corpus, in which the Korean phonemes were interpreted as similar English ones. Finally, the speech-adapted model was again trained with a tiny-scale Korean singing voice corpus for speech–singing adaptation. In the final adaptation, melodic supervision was chosen, which utilizes pitch information to improve the performance. For evaluation, the performance on multi-level error rates based on Word Error Rate (WER) was taken. Using the HuBERT-based transfer learning for adaptation improved the phoneme-level error rate of Korean speech by as much as 31.19%. Again, on singing voices by melodic supervision, it improved the rate by 0.55%. The significant improvement in speech recognition underscores the considerable potential of a model equipped with general human voice representations captured from the English corpus that can improve phoneme recognition on less target speech data. Moreover, the musical variation in singing voices is beneficial for phoneme recognition in singing voices. The proposed method could be applied to the phoneme recognition of other languages that have less speech and singing voice corpora.

A Longitudinal Exploration of Perception and Production of English Codas in CLIL Settings

Second language speech perception and production remain an enduring concern in second language acquisition, as research evidence seems to suggest that there is not a straightforward correspondence between these two speech domains and that their interrelationship seems to be of a complex nature. The present proposal intends to contribute to the inspection of such a relationship by observing the development of perception and production skills of English codas longitudinally in a group of secondary school learners in Spain involved in a content and language integrated learning (CLIL) program, which increases exposure and production opportunities. Results point to a slight overall improvement of both sound perception and production skills during a two-year period, the coda sounds exhibiting variable realizations. Many coda sounds were found to be identified and produced at near/ceiling levels while other codas remained at less successful identification and production levels even after two years of CLIL exposure. The correlation analyses performed indicated that the two dimensions tended to correlate when the development for each coda sound was inspected. No correlations were found when students’ individual overall performance in each dimension were examined, attesting individual differences.

Recognition of target domain Japanese speech using language model replacement

End-to-end (E2E) automatic speech recognition (ASR) models, which consist of deep learning models, are able to perform ASR tasks using a single neural network. These models should be trained using a large amount of data; however, collecting speech data which matches the targeted speech domain can be difficult, so speech data is often used that is not an exact match to the target domain, resulting in lower performance. In comparison to speech data, in-domain text data is much easier to obtain. Thus, traditional ASR systems use separately trained language models and HMM-based acoustic models. However, it is difficult to separate language information from an E2E ASR model because the model learns both acoustic and language information in an integrated manner, making it very difficult to create E2E ASR models for specialized target domain which are able to achieve sufficient recognition performance at a reasonable cost. In this paper, we propose a method of replacing the language information within pre-trained E2E ASR models in order to achieve adaptation to a target domain. This is achieved by deleting the “implicit” language information contained within the ASR model by subtracting the source-domain language model trained with a transcription of the ASR’s training data in a logarithmic domain. We then integrate a target domain language model through addition in the logarithmic domain. This subtraction and addition to replace of the language model is based on Bayes’ theorem. In our experiment, we first used two datasets of the Corpus of Spontaneous Japanese (CSJ) to evaluate the effectiveness of our method. We then we evaluated our method using the Japanese Newspaper Article Speech (JNAS) and CSJ corpora, which contain audio data from the read speech and spontaneous speech domain, respectively, to test the effectiveness of our proposed method at bridging the gap between these two language domains. Our results show that our proposed language model replacement method achieved better ASR performance than both non-adapted (baseline) ASR models and ASR models adapted using the conventional Shallow Fusion method.

Advantages and Pitfalls of Dataset Condensation: An Approach to Keyword Spotting with Time-Frequency Representations

With the exponential growth of data, the need for efficient techniques to extract relevant information from datasets becomes increasingly imperative. Reducing the training data can be useful for applications wherein storage space or computational resources are limited. In this work, we explore the concept of data condensation (DC) in the context of keyword spotting systems (KWS). Using deep learning architectures and time-frequency speech representations, we have obtained condensed speech signal representations using gradient matching with Efficient Synthetic-Data Parameterization. From a series of classification experiments, we analyze the models and condensed data performances in terms of accuracy and number of data per class. We also present results using cross-model techniques, wherein models are trained with condensed data obtained from a different architecture. Our findings demonstrate the potential of data condensation in the context of the speech domain for reducing the size of datasets while retaining their most important information and maintaining high accuracy for the model trained with the condensed dataset. We have obtained an accuracy of 80.75% with 30 condensed speech representations per class with ConvNet, representing an addition of 24.9% in absolute terms when compared to 30 random samples from the original training dataset. However, we demonstrate the limitations of this approach in the cross-model tests. We also highlight the challenges and opportunities for further improving the accuracy of condensed data obtained and trained with different neural network architectures.

Automatic Lyric Transcription and Automatic Music Transcription from Multimodal Singing

Automatic lyric transcription (ALT) refers to transcribing singing voices into lyrics, while automatic music transcription (AMT) refers to transcribing singing voices into note events, i.e., musical MIDI notes. Despite these two tasks having significant potential for practical application, they are still nascent. This is because the transcription of lyrics and note events solely from singing audio is notoriously difficult due to the presence of noise contamination, e.g., musical accompaniment, resulting in a degradation of both the intelligibility of sung lyrics and the recognizability of sung notes. To address this challenge, we propose a general framework for implementing multimodal ALT and AMT systems. Additionally, we curate the first multimodal singing dataset, comprising N20EMv1 and N20EMv2, which encompasses audio recordings and videos of lip movements, together with ground truth for lyrics and note events. For model construction, we propose adapting self-supervised learning models from the speech domain as acoustic encoders and visual encoders to alleviate the scarcity of labeled data. We also introduce a residual cross-attention mechanism to effectively integrate features from the audio and video modalities. Through extensive experiments, we demonstrate that our single-modal systems exhibit state-of-the-art performance on both ALT and AMT tasks. Subsequently, through single-modal experiments, we also explore the individual contributions of each modality to the multimodal system. Finally, we combine these and demonstrate the effectiveness of our proposed multimodal systems, particularly in terms of their noise robustness.

End-to-end integration of speech separation and voice activity detection for low-latency diarization of telephone conversations

Recent works show that speech separation guided diarization (SSGD) is an increasingly promising direction, mainly thanks to the recent progress in speech separation. It performs diarization by first separating the speakers and then applying voice activity detection (VAD) on each separated stream. In this work we conduct an in-depth study of SSGD in the conversational telephone speech (CTS) domain, focusing mainly on low-latency streaming diarization applications. We consider three state-of-the-art speech separation (SSep) algorithms and study their performance both in online and offline scenarios, considering non-causal and causal implementations as well as continuous SSep (CSS) windowed inference. We compare different SSGD algorithms on two widely used CTS datasets: CALLHOME and Fisher Corpus (Part 1 and 2) and evaluate both separation and diarization performance. To improve performance, a novel, causal and computationally efficient leakage removal algorithm is proposed, which significantly decreases false alarms. We also explore, for the first time, fully end-to-end SSGD integration between SSep and VAD modules. Crucially, this enables fine-tuning on real-world data for which oracle speakers sources are not available. In particular, our best model achieves 8.8% DER on CALLHOME, which outperforms the current state-of-the-art end-to-end neural diarization model, despite being trained on an order of magnitude less data and having significantly lower latency, i.e., 0.1 vs. 1 s. Finally, we also show that the separated signals can be readily used also for automatic speech recognition, reaching performance close to using oracle sources in some configurations.

Improved mini-batch multiple augmentation for low-resource spoken word recognition

Data augmentation techniques have been useful in dealing with limited data for machine learning tasks. Recently, spectrogram data augmentation techniques have been investigated for voice conversion and sound classification tasks and have produced better results. However, applying multiple data augmentation techniques within a mini-batch has been observed to lead to performance degradation. While applying multiple augmentation methods sequentially has shown performance gains in image data, transferring this approach to spectrogram data leads to loss of acoustic information. Hence, an alternative approach is needed to effectively utilize multiple augmentation methods in the speech domain. This study addressed these challenges in low-resource settings for spoken word recognition within the mini-batch. First, we investigated the effect of data augmentation techniques. Second, we investigated the effect of multiple data augmentation techniques. Finally, we proposed a new approach that uses an alternate mechanism to utilize multiple spectrogram augmentation techniques more effectively. The results of our experiment show that the proposed approach (new pattern) outperforms the sequential approach (traditional pattern) significantly at different scales of datasets, including low-resource settings. In addition, the proposed approach achieves approximately 2x actual speedup over the sequential approach. A combination of frequency-warping and time length control augmentation methods was found to be stable and robust in performance across all datasets evaluated.

Elective neck dissection versus elective neck irradiation in cT3/4N0 maxillary sinus squamous cell carcinoma: a propensity score matching analysis

BackgroundMaxillary sinus squamous cell carcinoma (MS-SCC) is an infrequent malignancy, and determining the optimal neck management for patients with cT3/4N0 MS-SCC remains a topic of ongoing debate. The purpose of this study was to compare the prognoses and quality of life outcomes of patients who underwent either elective neck dissection (END) or elective neck irradiation (ENI) for cT3/4N0 MS-SCC.MethodsIn this retrospective study, we enrolled patients with surgically treated cT3/4N0 MS-SCC, and the impact of different neck management strategies on regional control and disease-specific survival was compared using propensity score matching. The effect of surgical intervention on quality of life was evaluated using the Mann-Whitney U test.ResultsOf the 120 patients included, 36 underwent END. After propensity score matching, our analysis indicated that END did not lead to superior outcomes than ENI, as demonstrated by comparable rates of regional control (p = 0.990) and disease-specific survival (p = 0.999). However, in the 70 returned questionnaires, patients who underwent END reported higher scores in the domains of appearance, chewing, and speech than did patients who underwent ENI.ConclusionsOur findings suggest that while END and ENI contribute to similar prognoses, END yields superior functional outcomes.

IJCM_83A: Primary Ciliary Dyskinesia in a child

Background: PCD is rare genetic disorder with incidence of 1:10,000. We present a 10-year-boy who came with recurrent respiratory-infections with Dextrocardia and bilateral Bronchiectasis for interesting clinical presentation. Objective: PCD is rare inherited disease with mutations which impair normal cilia function, characterized by impaired muco-ciliary clearance, causing recurrent-respiratory infections leading to bronchiectasis. Treatment includes preventing and managing complications. Methodology: 10-year-old child came cough-cold since 2 days, and hurried breathing since 1 day. Child had multiple episodes of respiratory- infections which was treated with antibiotics and nebulization on OP basis. Developmental delay also noted, currently in speech domain, can speak 2-3 words. On examination, open mouth breathing and clubbing(grade 3) present. On auscultation bilateral bronchial breath sounds with coarse crepitations. Heart sounds heard louder on right side, but no murmur was heard. Chest X-ray and 2D-ECHO showed Situs-Solitus with Dextrocardia. Child required non-invasive ventilation and IV antibiotics for treatment. Child was weaned off to room air by day 4. CT Thorax done showed bronchiectatic changes in bilateral lung with volume loss of right hemithorax with mediastinal shift to right. CT Nasopharynx and OMC done for open-mouth breathing was normal, and advised steroid nasal spray for nasal blockage. Physiotherapist counselled parents about postural drainage. Genetic testing for PCD was sent, report awaited. He is on regular follow up and doing well. Results: Primary defect in PCD is structure and/or function of cilia leading to accumulation of secretions. Diagnostic criteria includes recurrent chest infections, bronchitis, and rhinitis, with situs-inversus, infertility, decreased mucociliary clearance and cilia showing ultrastructural defect on electron microscopy. We suspected PCD with clinical features and imaging studies. Conclusion: Patients are frequently troubled by repeated infections for which they need to seek medical attention which is reason for morbidity. As there is no specific treatment, early diagnosis and prevention of complication is key to management.

¿Notas La Diferencia? [Do You Hear the Difference?]: Perceptual Consequences of Intensive Voice Treatment in Spanish Speakers With Parkinson's Disease.

The primary objective of this study was to explore the effects of intensive voice-focused treatment on speech parameters in Spanish speakers with dysarthria associated with Parkinson's disease (PD) as perceived by naïve listeners. Fifteen Spanish speakers with dysarthria associated with PD received the Lee Silverman Voice Treatment (LSVT LOUD) for a month. Voice and speech recordings were conducted pretreatment, posttreatment, and at a 1-month follow-up. Thirty naïve adult listeners rated the perceptual dimensions of ease of understanding (EoU), resonance, articulatory precision, prosody, and voice quality from sentences extracted from an emotional monologue on a visual analogue scale. EoU, resonance, articulatory precision, and voice quality significantly improved pre- to posttreatment, but gains were not maintained at follow-up. Speech severity was a significant source of variance in mean listener response for all perceptual dimensions, although the interaction between speech severity and time was only significant for resonance and voice quality. LSVT LOUD may be beneficial to improve perceptual speech domains affected by PD in Spanish speakers with dysarthria. Its impact on the different speech subsystems may reflect a universal distribution of effects when directly targeting the glottal source. Language-specific contributions of each perceptual domain to speech intelligibility should be explored in further research to determine linguistically sensitive treatment targets.

Specificity of Motor Contributions to Auditory Statistical Learning.

Statistical learning is the ability to extract patterned information from continuous sensory signals. Recent evidence suggests that auditory-motor mechanisms play an important role in auditory statistical learning from speech signals. The question remains whether auditory-motor mechanisms support such learning generally or in a domain-specific manner. In Experiment 1, we tested the specificity of motor processes contributing to learning patterns from speech sequences. Participants either whispered or clapped their hands while listening to structured speech. In Experiment 2, we focused on auditory specificity, testing whether whispering equally affects learning patterns from speech and non-speech sequences. Finally, in Experiment 3, we examined whether learning patterns from speech and non-speech sequences are correlated. Whispering had a stronger effect than clapping on learning patterns from speech sequences in Experiment 1. Moreover, whispering impaired statistical learning more strongly from speech than non-speech sequences in Experiment 2. Interestingly, while participants in the non-speech tasks spontaneously synchronized their motor movements with the auditory stream more than participants in the speech tasks, the effect of the motor movements on learning was stronger in the speech domain. Finally, no correlation between speech and non-speech learning was observed. Overall, our findings support the idea that learning statistical patterns from speech versus non-speech relies on segregated mechanisms, and that the speech motor system contributes to auditory statistical learning in a highly specific manner.

Analyzing sentiment and its application in deep learning: Consistent behavior across multiple occasions

This article offers a systematic review of the evolution in sentiment analysis techniques, moving from unimodal to multimodal to multi-occasion methodologies, with an emphasis on the integration and application of deep learning in sentiment analysis. Firstly, the paper presents the theoretical foundation of sentiment analysis, including the definition and classification of affect and emotion. It then delves into the pivotal technologies used in unimodal sentiment analysis, specifically within the domains of text, speech, and image analysis, examining feature extraction, representation, and classification models. Subsequently, the focus shifts to multimodal sentiment analysis. The paper offers a survey of widely utilized multimodal sentiment datasets, feature representation and fusion techniques, as well as deep learning-based multimodal sentiment analysis models such as attention networks and graph neural networks. It further addresses the application of these multimodal sentiment analysis techniques in social media, product reviews, and public opinion monitoring. Lastly, the paper underscores that challenges persist in the area of multimodal sentiment fusion, including data imbalance and disparities in feature expression. It calls for further research into cross-modal feature expression, dataset augmentation, and explainable modeling to enhance the performance of complex sentiment analysis across multiple occasions.

Time domain speech enhancement with CNN and time-attention transformer

Speech enhancement in the time domain involves improving the quality and intelligibility of noisy speech by processing the waveform directly without the need for explicit feature extraction or domain transformation. Deep learning is a powerful approach for time domain speech enhancement, offering significant improvements over traditional techniques. Formulating a resource-efficient deep neural model in the time domain without ignoring the contextual information and detailed features of input speech is still a vital challenge. To address this challenge, this study proposes a speech enhancement model using 1D-time domain dilated residual blocks in the convolutional encoder-decoder framework. Further, this study integrates a time-attention transformer (TAT) bottleneck between the encoder-decoder. The TAT model extends the transformer architecture by incorporating a time-attention mechanism, which enables the model to selectively attend to different segments of the speech signal over time. This allows the model to effectively capture long-term dependencies in the speech and learn to recognize important features. The experimental results indicate that the proposed speech enhancement outperforms the recent deep neural networks (DNNs) and substantially improves the intelligibility and quality of noisy speech. With the WSJ0 SI-84 database, the proposed SE improves the STOI and PESQ by 21.51% and 1.14 over noisy speech.

An Investigation of Quality of Life of Head-and-neck Cancer Patients after Tumor Resection and Subsequent Prosthetic Rehabilitation: A Clinical Study

ABSTRACT Background: Head-and-neck cancer and its treatment can decrease the quality of life (QoL) of patients in consequence of the loss of structural and functional integrity of the oro- and nasopharyngeal region. Maxillofacial prosthetic treatment is advised when anatomical structures of the head and neck are not replaceable by living tissue, when recurrence is likely, when radiotherapy is administered, or when fragment of fractured bones are severely displaced. The aim of the study was to examine whether prosthetic rehabilitation of maxillofacial defects affects the QoL of patients using the University of Washington (UW)-QoL. Materials and Methods: This was the prospective interventional study using pre- and post-design using a convenience sampling method. The prosthesis was fabricated as per the clinical situation using heat-polymerized polymethyl methacrylate, maxillofacial silicone, or a combination of both. Results: The overall mean composite QOL score before rehabilitation was 68.05 ± 16.98 and after rehabilitation, it was 73.22 ± 11.28. Chewing and saliva (dryness of the mouth) had the lowest scores (55 ± 9.2, 62 ± 8.6, respectively) among all domains, whereas pain and anxiety had the highest scores (93 ± 3.7, 95 ± 1.5, respectively). The domains of appearance, speech, swallowing, chewing, saliva, and overall QoL showed statistically significant improvements postprosthetic rehabilitation. Conclusions: In the present study and selected population, the subjective analysis using UW-QOL showed that despite altered anatomy following resection and reconstruction, compromised physiology following chemoradiation, the oral cancer patients coped well and adapted to near-normal oral status after prosthetic rehabilitation. This was evident from their improved QOL after 1 month of prosthetic rehabilitation.

Real-Time Subtitle Generator for Sinhala Speech

In today’s digital era, the significance of speech recognition technology cannot be overstated as it plays a pivotal role in enabling human-computer interaction and supporting various applications. This paper focuses on the development of a real-time subtitle generator for Sinhala speech using speech recognition techniques. The CMUSphinx toolkit, an open-source toolkit based on the Hidden Markov Model (HMM), is employed for the implementation of the application. Mel-frequency cepstral coefficients (MFCC) are utilized for feature extraction from the given ’wav’ format recordings. The paper places significant emphasis on the importance of a real-time subtitle generator for Sinhala speech and explores the existing literature in the field. It outlines the objectives of the research and discusses the achieved outcomes. By fine-tuning hyperparameters to enhance the recognition accuracy of the system, impressive results of 88.28% training accuracy and 11.72% Word Error Rate (WER) are attained. Thesignificance of this research is underscored by its methodological advancements, robust performance metrics, and the potential impact on facilitating seamless interactions and applications in the Sinhala speech domain.
 Keywords: Speech recognition, Real-time, Subtitle, CMUSphinx, Open source, Hidden Markov Model, Mel-frequency cepstral coefficients, ’wav’, Accuracy, Word Error Rate

Optimising speech‐testing to predict prodromal Alzheimer’s disease: head‐to‐head comparison study of tasks and analysis methods

AbstractBackgroundChanges in speech occur in early‐stage Alzheimer’s disease. A range of approaches have been used for eliciting and automatically analysing speech, but limited research has directly compared these methods.MethodParticipants from the AMYPRED‐UK (NCT04828122) and AMYPRED‐US (NCT04928976) studies completed four speech‐elicitation tasks: the Automatic Story Recall Task (ASRT), the Logical Memory Test (LMT), Semantic (Animals, Vegetables, Fruit) and Phonemic Verbal Fluency (F, A, S) Tasks. Responses were recorded and automatically transcribed. Analyses were completed with Novoic’s speech analysis software, using four key approaches: (1) feature extraction, (2) representations from large language models (LLM), (3) text‐similarity evaluation, and (4) autoscoring. Together, these evaluated audio, linguistic, and temporal speech domains. Outputs were entered into logistic regression models predicting Mild Cognitive Impairment (MCI)/mild AD and Amyloid positivity in MCI/mild AD. Area under the ROC curves (AUC) were evaluated via 5‐fold cross‐validation.Result165 older adults (including N = 74 MCI and 9 mild AD; of which N = 38 MCI/mild AD and amyloid beta positive) provided data for all tasks. Strength of model predictions varied by task, analytic approach and domain (Fig 1). AUCs predicting MCI/mild AD were generally higher for ASRTs compared to other tasks (Fig 1A). Text‐similarity approaches (G‐match: similarity of word embeddings between the source text and retelling, and V‐match: string‐based matching), produced the highest AUC for ASRTs (G‐match AUC = 0.87‐0.88), and the LMT (V‐match AUC = 0.83‐85). Autoscoring tasks produced the strongest predictor for the semantic fluency (AUC = 0.85), and also one of the strongest for phonemic fluency (AUC = 0.72). Changes in linguistic and temporal characteristics were more sensitive to MCI/mild AD, compared to audio‐only metrics. Amyloid in MCI/mild AD predictions were overall less strong (Fig 1B), with the most consistent pattern being better performance of text‐similarity approaches on story recall tasks (up to AUC = 0.77 for ASRTs, 0.73 for LMT).ConclusionSelection of task and analytic approach is important for developing more sensitive speech‐based testing. The best performing tasks (ASRT, Semantic Fluency) and metrics (text‐similarity metrics and autoscoring) have been incorporated in Novoic’s Storyteller automated remote speech‐based test.

Optimising speech‐testing to predict prodromal Alzheimer’s disease: head‐to‐head comparison study of tasks and analysis methods

AbstractBackgroundChanges in speech occur in early‐stage Alzheimer’s disease. A range of approaches have been used for eliciting and automatically analysing speech, but limited research has directly compared these methods.MethodParticipants from the AMYPRED‐UK (NCT04828122) and AMYPRED‐US (NCT04928976) studies completed four speech‐elicitation tasks: the Automatic Story Recall Task (ASRT), the Logical Memory Test (LMT), Semantic (Animals, Vegetables, Fruit) and Phonemic Verbal Fluency (F, A, S) Tasks. Responses were recorded and automatically transcribed. Analyses were completed with Novoic’s speech analysis software, using four key approaches: (1) feature extraction, (2) representations from large language models (LLM), (3) text‐similarity evaluation, and (4) autoscoring. Together, these evaluated audio, linguistic, and temporal speech domains. Outputs were entered into logistic regression models predicting Mild Cognitive Impairment (MCI)/mild AD and Amyloid positivity in MCI/mild AD. Area under the ROC curves (AUC) were evaluated via 5‐fold cross‐validation.Result165 older adults (including N = 74 MCI and 9 mild AD; of which N = 38 MCI/mild AD and amyloid beta positive) provided data for all tasks. Strength of model predictions varied by task, analytic approach and domain (Fig 1). AUCs predicting MCI/mild AD were generally higher for ASRTs compared to other tasks (Fig 1A). Text‐similarity approaches (G‐match: similarity of word embeddings between the source text and retelling, and V‐match: string‐based matching), produced the highest AUC for ASRTs (G‐match AUC = 0.87‐0.88), and the LMT (V‐match AUC = 0.83‐85). Autoscoring tasks produced the strongest predictor for the semantic fluency (AUC = 0.85), and also one of the strongest for phonemic fluency (AUC = 0.72). Changes in linguistic and temporal characteristics were more sensitive to MCI/mild AD, compared to audio‐only metrics. Amyloid in MCI/mild AD predictions were overall less strong (Fig 1B), with the most consistent pattern being better performance of text‐similarity approaches on story recall tasks (up to AUC = 0.77 for ASRTs, 0.73 for LMT).ConclusionSelection of task and analytic approach is important for developing more sensitive speech‐based testing. The best performing tasks (ASRT, Semantic Fluency) and metrics (text‐similarity metrics and autoscoring) have been incorporated in Novoic’s Storyteller automated remote speech‐based test.