Set Of Acoustic Features Research Articles

Exploring corpus-invariant emotional acoustic feature for cross-corpus speech emotion recognition

Unsupervised cross-corpus speech emotion recognition (SER) is the task where the labeled training (source) and unlabeled testing (target) speech come from different corpora. Subspace transfer learning is one of the mainstream technologies for tackling cross-corpus SER challenges, which is achieved by utilizing projection matrices to learn common corpus-invariant feature representations between the source and target corpus. However, these methods mainly focus on mapping modeling from the input low-level descriptor (LLD) feature space to the corpus-invariant emotional feature space, which is an implicit feature mapping process that lacks interpretability for the selected features. This omission leads to an inability to pinpoint which acoustic features possess corpus invariance. To bridge this gap, we first propose a new transfer subspace learning framework with feature selection capabilities, i.e., the Corpus-Invariant Emotional Acoustic Feature Seeker (CAFS). Specifically, the CAFS integrates two core terms into the transfer regression loss function: (1) the emotion preservation term: This term includes emotional regression and the l2,1 norm, which is mainly used to select features and ensure that these features are related to emotions. (2) the corpus invariance preservation term: This item is mainly used to measure the difference in feature distribution between the source and target corpora. Minimizing this term bridges the gap between the source and target domains, ensuring that the chosen acoustic features are corpus-invariant. Subsequently, we conducted extensive cross-corpus SER experiments to explore corpus-invariant emotional acoustic features under various commonly used acoustic feature sets (IS09 and eGeMAPS). Through statistical analysis of the acoustic features sought by the CAFS framework, some acoustic features (e.g., Mel-Frequency Cepstral Coefficients (MFCC) and Formant) reveal their corpus-invariant properties, which could provide insights for feature selection in cross-corpus SER. These findings also lay the groundwork for a solid theoretical and empirical foundation for future research and applications in cross-corpus SER.

Digital voice-based biomarker for monitoring respiratory quality of life: findings from the colive voice study

Regular monitoring of respiratory quality of life (RQoL) is essential in respiratory healthcare, facilitating prompt diagnosis and tailored treatment for chronic respiratory diseases. Voice alterations resulting from respiratory conditions create unique audio signatures that can potentially be utilized for disease screening or monitoring. Analyzing data from 1908 participants from the Colive Voice study, which collects standardized voice recordings alongside comprehensive demographic, epidemiological, and patient-reported outcome data, we evaluated various strategies to estimate RQoL from voice, including handcrafted acoustic features, standard acoustic feature sets, and advanced deep audio embeddings derived from pretrained convolutional neural networks. We compared models using clinical features alone, voice features alone, and a combination of both. The multimodal model combining clinical and voice features demonstrated the best performance, achieving an accuracy of 70.8% and an area under the receiver operating characteristic curve (AUROC) of 0.77; an improvement of over 5% in terms of accuracy and 7% in terms of AUROC compared to model utilizing voice features alone. Incorporating vocal biomarkers significantly enhanced the predictive capacity of clinical variables across all acoustic feature types, with a net classification improvement (NRI) of up to 0.19. Our digital voice-based biomarker is capable of accurately predicting RQoL, either as an alternative to or in conjunction with clinical measures, and could be used to facilitate rapid screening and remote monitoring of respiratory health status.

Pre-trained models for detection and severity level classification of dysarthria from speech

Automatic detection and severity level classification of dysarthria from speech enables non-invasive and effective diagnosis that helps clinical decisions about medication and therapy of patients. In this work, three pre-trained models (wav2vec2-BASE, wav2vec2-LARGE, and HuBERT) are studied to extract features to build automatic detection and severity level classification systems for dysarthric speech. The experiments were conducted using two publicly available databases (UA-Speech and TORGO). One machine learning-based model (support vector machine, SVM) and one deep learning-based model (convolutional neural network, CNN) was used as the classifier. In order to compare the performance of the wav2vec2-BASE, wav2vec2-LARGE, and HuBERT features, three popular acoustic feature sets, namely, mel-frequency cepstral coefficients (MFCCs), openSMILE and extended Geneva minimalistic acoustic parameter set (eGeMAPS) were considered. Experimental results revealed that the features derived from the pre-trained models outperformed the three baseline features. It was also found that the HuBERT features performed better than the wav2vec2-BASE and wav2vec2-LARGE features. In particular, when compared to the best-performing baseline feature (openSMILE), the HuBERT features showed in the detection problem absolute accuracy improvements that varied between 1.33% (the SVM classifier, the TORGO database) and 2.86% (the SVM classifier, the UA-Speech database). In the severity level classification problem, the HuBERT features showed absolute accuracy improvements that varied between 6.54% (the SVM classifier, the TORGO database) and 10.46% (the SVM classifier, the UA-Speech database) compared to the best-performing baseline feature (eGeMAPS).

Detecting bulbar amyotrophic lateral sclerosis (ALS) using automatic acoustic analysis

Automatic speech assessments have the potential to dramatically improve ALS clinical practice and facilitate patient stratification for ALS clinical trials. Acoustic speech analysis has demonstrated the ability to capture a variety of relevant speech motor impairments, but implementation has been hindered by both the nature of lab-based assessments (requiring travel and time for patients) and also by the opacity of some acoustic feature analysis methods. These challenges and others have obscured the ability to distinguish different ALS disease stages/severities. Validation of automated acoustic analysis tools could enable detection of early signs of ALS, and these tools could be deployed to screen and monitor patients without requiring clinic visits. Here, we sought to determine whether acoustic features gathered using an automated assessment app could detect ALS as well as different levels of speech impairment severity resulting from ALS. Speech samples (readings of a standardized, 99-word passage) from 119 ALS patients with varying degrees of disease severity as well as 22 neurologically healthy participants were analyzed, and 53 acoustic features were extracted. Patients were stratified into early and late stages of disease (ALS-early/ALS-E and ALS-late/ALS-L) based on the ALS Functional Ratings Scale-Revised bulbar score (FRS-bulb) (median [interquartile range] of FRS-bulbar scores: 11[3]). The data were analyzed using a sparse Bayesian logistic regression classifier. It was determined that the current relatively small set of acoustic features could distinguish between ALS and controls well (area under receiver-operating characteristic curve/AUROC = 0.85), that the ALS-E patients could be separated well from control participants (AUROC = 0.78), and that ALS-E and ALS-L patients could be reasonably separated (AUROC = 0.70). These results highlight the potential for automated acoustic analyses to detect and stratify ALS.

EARLY DETECTION OF ALZHEIMER’S DISEASE AND RELATED DEMENTIAS FROM VOICE RECORDINGS: THE FRAMINGHAM HEART STUDY

Abstract With the aging global population and the increasing prevalence of dementia, there is a growing focus on identifying mild cognitive impairment (MCI), a pre-dementia state, to enable timely interventions that could potentially slow down neurodegeneration. Producing speech is a cognitively complex task that engages various cognitive domains, while the ease of audio data collection underscores the potential cost-effectiveness and noninvasiveness that voice may offer. This study aims to construct a machine learning pipeline that incorporates speaker diarization, feature extraction, feature selection, and classification, to identify a set of acoustic features that exhibit strong MCI detection capability. The study included 100 MCI cases and 100 healthy controls (HC) matched for age, sex, and education from the Framingham Heart Study. Participants’ speech responses during cognitive tests were recorded, and the recorded audio was processed to identify segments of each participant’s voice from recordings that included voices of both testers and participants. A comprehensive set of 6385 acoustic features was then extracted from these voice segments using the OpenSMILE and Praat softwares. Subsequently, we constructed a random forest model using the features that exhibited significant differences between the MCI and HC groups. We identified an optimal subset of 29 features that resulted in an AUC of 0.87, with a 90% confidence interval ranging from 0.82 to 0.93. This study showcases the potential of human voice as a valuable resource for improving early detection of ADRD and motivates future opportunities to use passive voice collection tools, such as hearing aids, to measure brain health.

An Online Attachment Style Recognition System Based on Voice and Machine Learning.

Attachment styles are known to have significant associations with mental and physical health. Specifically, insecure attachment leads individuals to higher risk of suffering from mental disorders and chronic diseases. The aim of this study is to develop an attachment recognition model that can distinguish between secure and insecure attachment styles from voice recordings, exploring the importance of acoustic features while also evaluating gender differences. A total of 199 participants recorded their responses to four open questions intended to trigger their attachment system using a web-based interrogation system. The recordings were processed to obtain the standard acoustic feature set eGeMAPS, and recursive feature elimination was applied to select the relevant features. Different supervised machine learning models were trained to recognize attachment styles using both gender-dependent and gender-independent approaches. The gender-independent model achieved a test accuracy of 58.88%, whereas the gender-dependent models obtained 63.88% and 83.63% test accuracy for women and men respectively, indicating a strong influence of gender on attachment style recognition and the need to consider them separately in further studies. These results also demonstrate the potential of acoustic properties for remote assessment of attachment style, enabling fast and objective identification of this health risk factor, and thus supporting the implementation of large-scale mobile screening systems.

VocDoc, what happened to my voice? Towards automatically capturing vocal fatigue in the wild

Objective:Voice problems that arise during everyday vocal use can hardly be captured by standard outpatient voice assessments. In preparation for a digital health application to automatically assess longitudinal voice data ‘in the wild’ – the VocDoc, the aim of this paper was to study vocal fatigue from the speaker’s perspective, the healthcare professional’s perspective, and the ‘machine’s’ perspective. Methods:We collected data of four voice healthy speakers completing a 90-min reading task. Every 10 min the speakers were asked about subjective voice characteristics. Then, we elaborated on the task of elapsed speaking time recognition: We carried out listening experiments with speech and language therapists and employed random forests on the basis of extracted acoustic features. We validated our models speaker-dependently and speaker-independently and analysed underlying feature importances. For an additional, clinical application-oriented scenario, we extended our dataset for lecture recordings of another two speakers. Results:Self- and expert-assessments were not consistent. With mean F1 scores up to 0.78, automatic elapsed speaking time recognition worked reliably in the speaker-dependent scenario only. A small set of acoustic features – other than features previously reported to reflect vocal fatigue – was found to universally describe long-term variations of the voice. Conclusion:Vocal fatigue seems to have individual effects across different speakers. Machine learning has the potential to automatically detect and characterise vocal changes over time. Significance:Our study provides technical underpinnings for a future mobile solution to objectively capture pathological long-term voice variations in everyday life settings and make them clinically accessible.

Leveraging machine learning and similarity judgements to identify perceptually relevant acoustic features of small, unmanned aircraft system signal similarity

Auditory signals can be described quantitatively by a set of measurable acoustic features (i.e., zero crossing rate, attack slope, etc.) or qualitatively with adjectives, such as whooping, thunderous, melodic, or in comparative terms such as different, louder, etc. Listeners can rate the similarity of signals and assign qualitative descriptions relatively easily; however, most lack the ability to articulate the quantitative basis of these judgments. Because the qualitative differences in signals typically correlate to a measurable difference in the acoustic features, signal similarity ratings can be used to recover the acoustic features that define signal similarity. In the present study, subjects were given combinations of small, unmanned aircraft systems (SUAS) signals consisting of either two different SUASs or SUAS and non-SUAS and asked to rate similarity on a scale from non-similar to highly similar. Using these similarity ratings along with acoustic difference features, machine learning algorithms were trained to predict human responses. These algorithms predict the position of a withheld UAS signal within the similarity feature-space. Crucial prediction acoustic difference features are extracted from the algorithms via feature importance and sensitivity analysis techniques. The extracted acoustic difference features may be inferred as prominent information impacting the human perception of signal similarity.

Assessment of radio(chemo)therapy-related dysphagia in head and neck cancer patients based on cough-related acoustic features: a prospective phase II national clinical trial (ACCOUGH-P/A trial)

BackgroundRadiation-associated dysphagia is defined as impaired swallowing efficiency/safety following (chemo)radiotherapy in head and neck cancer patients. In a dysphagia framework, impaired coughing may lead to lung aspiration and fatal lung infection. Although cough efficacy is a predictor of the risk of aspiration, cough investigation is minimal in patients with radiation-associated dysphagia. Because cough is a transient signal, existing software for speech analysis are not appropriate. The goal of our project is to develop an assessment method using acoustic features related to voluntary and reflexive coughs as biomarkers of the risk of penetration/aspiration in patients with radiation-associated dysphagia.MethodsHealthy subjects and head and neck cancer patients with and without dysphagia will produce voluntary coughs, throat clearings and reflexive coughs. Recordings will be made using an acoustic microphone and a throat microphone. The recorded signals will be manually segmented and subsequently analysed with a software under development. Automatic final segmentation enables to measure cough duration. The first method of analysis includes temporal features: the amplitude contour, the sample entropy and the kurtosis. These features report respectively the strength, the unpredictability (turbulence noise due to the air jet) and the impulsive quality (burst) of the signal. The second method of analysis consists of a spectral decomposition of the relative cough signal energy into several frequency bands (0–400 Hz, 400–800 Hz, 800–1600 Hz, 1600–3200 Hz, > 3200 Hz). The primary outcome of this exploratory research project is the identification of a set of descriptive acoustic cough features in healthy subjects as reference data (ACCOUGH). The secondary outcome of this research in head and neck cancer patients with radiation-associated dysphagia includes the identification of (1) a set of descriptive acoustic cough features as biomarkers of penetration-aspiration (ACCOUGH-P/A), (2) swallowing scores, (3) voice features and (4) aerodynamic cough features.DiscussionThis study is expected to develop methods of acoustic cough analysis to enhance the assessment of radiation-associated dysphagia in head and neck cancer patients following (chemo)radiation.Trial registrationInternational Standard Randomized Controlled Trials Number (ISRCTN) registry ISRCTN16540497. Accepted on 23 June 2023.

Evaluating the clinical utility of speech analysis and machine learning in schizophrenia: A pilot study

BackgroundSchizophrenia is a serious mental disorder that significantly impacts social functioning and quality of life. However, current diagnostic methods lack objective biomarker support. While some studies have indicated differences in audio features between patients with schizophrenia and healthy controls, these findings are influenced by demographic information and variations in experimental paradigms. Therefore, it is crucial to explore stable and reliable audio biomarkers for an auxiliary diagnosis and disease severity prediction of schizophrenia. MethodA total of 130 individuals (65 patients with schizophrenia and 65 healthy controls) read three fixed texts containing positive, neutral, and negative emotions, and recorded them. All audio signals were preprocessed and acoustic features were extracted by a librosa-0.9.2 toolkit. Independent sample t-tests were performed on two sets of acoustic features, and Pearson correlation on the acoustic features and Positive and Negative Syndrome Scale (PANSS) scores of the schizophrenia group. Classification algorithms in scikit-learn were used to diagnose schizophrenia and predict the level of negative symptoms. ResultsSignificant differences were observed between the two groups in the mfcc_8, mfcc_11, and mfcc_33 of mel-frequency cepstral coefficient (MFCC). Furthermore, a significant correlation was found between mfcc_7 and the negative PANSS scores. Through acoustic features, we could not only differentiate patients with schizophrenia from healthy controls with an accuracy of 0.815 but also predict the grade of the negative symptoms in schizophrenia with an average accuracy of 0.691. ConclusionsThe results demonstrated the considerable potential of acoustic characteristics as reliable biomarkers for diagnosing schizophrenia and predicting clinical symptoms.

Crocodile perception of distress in hominid baby cries.

It is generally argued that distress vocalizations, a common modality for alerting conspecifics across a wide range of terrestrial vertebrates, share acoustic features that allow heterospecific communication. Yet studies suggest that the acoustic traits used to decode distress may vary between species, leading to decoding errors. Here we found through playback experiments that Nile crocodiles are attracted to infant hominid cries (bonobo, chimpanzee and human), and that the intensity of crocodile response depends critically on a set of specific acoustic features (mainly deterministic chaos, harmonicity and spectral prominences). Our results suggest that crocodiles are sensitive to the degree of distress encoded in the vocalizations of phylogenetically very distant vertebrates. A comparison of these results with those obtained with human subjects confronted with the same stimuli further indicates that crocodiles and humans use different acoustic criteria to assess the distress encoded in infant cries. Interestingly, the acoustic features driving crocodile reaction are likely to be more reliable markers of distress than those used by humans. These results highlight that the acoustic features encoding information in vertebrate sound signals are not necessarily identical across species.

Fault Diagnosis of Train Wheelset Bearing Roadside Acoustics Considering Sparse Operation with GA-RBF

Trackside acoustic signals are useful for non-contact measurements as well as early warnings in the diagnosis of train wheelset bearing faults. However, there are two important problems when using roadside acoustic signals to diagnose wheel-to-wheel bearing faults; one is the presence of strong interference from strong noise and high harmonics in the signal, and the other is the low efficiency of bearing fault identification caused by it. Therefore, from the viewpoint of solving the two problems, a sparse operation method is proposed for denoising and detuning the modulation of the roadside acoustic signal, and a machine learning classifier with a Genetic Algorithm (GA)-optimized Radial Basis Neural Network (RBFNN) is proposed to improve the rate at which the features of roadside acoustic signal faults are recognized. Firstly, the background noise is filtered out from the Doppler-corrected acoustic signal using the Sparse Representation method, and the inverse wavelet transform is reconstructed into a noiseless signal. Secondly, the interference high-harmonic signal in the signal is filtered out using the Resonant Sparse Signal Decomposition (RSSD) method. Then, the GA is selected to optimize the parameters of the RBF neural network and build a fault diagnosis model. Finally, the extracted acoustic signal feature set is trained on the network model, and the trained model is used for testing. In summary, the sparse operation on the roadside acoustic signal processing and the GA-RBFNN diagnosis model were verified as being very effective in the diagnosis of roadside acoustic train wheel pair faults through the simulation experiment.

Association of acoustic perturbation measures with dementia status: A Framingham Heart Study

AbstractBackgroundDiagnosis of dementia due to Alzheimer’s disease (AD) is a resource‐intensive process, often requiring clinical assessments and neuroimaging which may fall beyond the reach of patients in remote settings. Given the challenges of traditional dementia work‐up, identification of disease‐relevant signatures from low‐cost modalities has attracted interest as a means to improve disease detection. Here, we describe a machine learning strategy to associate acoustic perturbations in voice with dementia status using digital recordings of neuropsychological exams administered in the Framingham Heart Study (FHS).MethodWe used 118 voice recordings from 79 participants to extract jitter, shimmer, and haronics‐to‐nosie ratio (HNR) from the ComParE 2016 dataset. Each of these acoustic perturbation measures are commonly used in the clinical context to characterize pathological voice. For each voice transcript, these features were computed and and fed as input to a one‐dimensional convolutional neural network to predict dementia status. The data was randomly split in the ratio of 1:1 (at the participant‐level), where 50% of the cases were used for training and the remaining for testing. This process was repeated 10 times and the average model performance was reported. For each model run, the neural network was trained for 500 epochs without early stopping and the model obtained after the final epoch was used for testing.ResultThe mean (±std) age of the participants included in the study was 82.6±8.3 years, and 21 of them were ApoE positive. The mean (±std) area under the receiver operating characteristic curve and the mean (±std) area under the precision‐recall curve of the model classifying participants with dementia from those with normal cognition was 0.673±0.043 and 0.791±0.052, respectively (Fig. 1). Also, the mean accuracy (±std) was 0.615±0.025, the mean precision (±std) was 0.714±0.121, and the mean sensitivity/recall (±std) was 0.721±0.179.ConclusionOur study presents a proof‐of‐principle that deep learning on acoustic perturbation measures can predict dementia status. Further studies with additional participants and an expanded set of acoustic, speech and linguistic features are needed to evaluate the full potential of machine learning to derive disease‐relevant signatures from audio data, and potentially expand the reach of early detection and intervention efforts.

Speech Entrainment in Adolescent Conversations: A Developmental Perspective.

Defined as the similarity of speech behaviors between interlocutors, speech entrainment plays an important role in successful adult conversations. According to theoretical models of entrainment and research on motoric, cognitive, and social developmental milestones, the ability to entrain should develop throughout adolescence. However, little is known about the specific developmental trajectory or the role of speech entrainment in conversational outcomes of this age group. The purpose of this study is to characterize speech entrainment patterns in the conversations of neurotypical early adolescents. This study utilized a corpus of 96 task-based conversations between adolescents between the ages of 9 and 14 years and a comparison corpus of 32 task-based conversations between adults. For each conversational turn, two speech entrainment scores were calculated for 429 acoustic features across rhythmic, articulatory, and phonatory dimensions. Predictive modeling was used to evaluate the degree of entrainment and relationship between entrainment and two metrics of conversational success. Speech entrainment increased throughout early adolescence but did not reach the level exhibited in conversations between adults. Additionally, speech entrainment was predictive of both conversational quality and conversational efficiency. Furthermore, models that included all acoustic features and both entrainment types performed better than models that only included individual acoustic feature sets or one type of entrainment. Our findings show that speech entrainment skills are largely developed during early adolescence with continued development possibly occurring across later adolescence. Additionally, results highlight the role of speech entrainment in successful conversation in this population, suggesting the import of continued exploration of this phenomenon in both neurotypical and neurodivergent adolescents. We also provide evidence of the value of using holistic measures that capture the multidimensionality of speech entrainment and provide a validated methodology for investigating entrainment across multiple acoustic features and entrainment types.

Efficient Pause Extraction and Encode Strategy for Alzheimer's Disease Detection Using Only Acoustic Features from Spontaneous Speech.

Clinical studies have shown that speech pauses can reflect the cognitive function differences between Alzheimer's Disease (AD) and non-AD patients, while the value of pause information in AD detection has not been fully explored. Herein, we propose a speech pause feature extraction and encoding strategy for only acoustic-signal-based AD detection. First, a voice activity detection (VAD) method was constructed to detect pause/non-pause feature and encode it to binary pause sequences that are easier to calculate. Then, an ensemble machine-learning-based approach was proposed for the classification of AD from the participants' spontaneous speech, based on the VAD Pause feature sequence and common acoustic feature sets (ComParE and eGeMAPS). The proposed pause feature sequence was verified in five machine-learning models. The validation data included two public challenge datasets (ADReSS and ADReSSo, English voice) and a local dataset (10 audio recordings containing five patients and five controls, Chinese voice). Results showed that the VAD Pause feature was more effective than common feature sets (ComParE: 6373 features and eGeMAPS: 88 features) for AD classification, and that the ensemble method improved the accuracy by more than 5% compared to several baseline methods (8% on the ADReSS dataset; 5.9% on the ADReSSo dataset). Moreover, the pause-sequence-based AD detection method could achieve 80% accuracy on the local dataset. Our study further demonstrated the potential of pause information in speech-based AD detection, and also contributed to a more accessible and general pause feature extraction and encoding method for AD detection.

Speech Emotion Recognition Based on Multiple Acoustic Features and Deep Convolutional Neural Network

Speech emotion recognition (SER) plays a vital role in human–machine interaction. A large number of SER schemes have been anticipated over the last decade. However, the performance of the SER systems is challenging due to the high complexity of the systems, poor feature distinctiveness, and noise. This paper presents the acoustic feature set based on Mel frequency cepstral coefficients (MFCC), linear prediction cepstral coefficients (LPCC), wavelet packet transform (WPT), zero crossing rate (ZCR), spectrum centroid, spectral roll-off, spectral kurtosis, root mean square (RMS), pitch, jitter, and shimmer to improve the feature distinctiveness. Further, a lightweight compact one-dimensional deep convolutional neural network (1-D DCNN) is used to minimize the computational complexity and to represent the long-term dependencies of the speech emotion signal. The overall effectiveness of the proposed SER systems’ performance is evaluated on the Berlin Database of Emotional Speech (EMODB) and the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) datasets. The proposed system gives an overall accuracy of 93.31% and 94.18% for the EMODB and RAVDESS datasets, respectively. The proposed MFCC and 1-D DCNN provide greater accuracy and outpace the traditional SER techniques.

SELECTION OF DISCRIMINATIVE FEATURES FOR ARABIC PHONEME’S MISPRONUNCIATION DETECTION

Pronunciation training is an important part of Computer Assisted Pronunciation Training (CAPT) systems. Mispronunciation detection systems recognized pronunciation mistakes from user’s speech and provided them feedback about their pronunciation. Acoustic phonetic features plays a vital role in speech classification based applications. This research work investigated the suitability of various acoustic features: pitch, energy, spectrum flux, zero-crossing, Entropy and MelFrequency Cepstral Coefficients (MFCCs). Sequential Forward Selection (SFS) was used to find out most suitable acoustic features from the computed feature set. This study used K-Nearest Neighbors (K-NN) classifier was used to detect the pronunciation mistakes from Arabic phonemes. This research selected the set of most discriminative acoustic features for each phoneme. K-NN achieved accuracy of 92.15% for mispronunciation detection of Arabic Phonemes.

SELECTION OF DISCRIMINATIVE FEATURES FOR ARABIC PHONEME’S MISPRONUNCIATION DETECTION

Pronunciation training is an important part of Computer Assisted Pronunciation Training (CAPT) systems. Mispronunciation detection systems recognized pronunciation mistakes from user’s speech and provided them feedback about their pronunciation. Acoustic phonetic features plays a vital role in speech classification based applications. This research work investigated the suitability of various acoustic features: pitch, energy, spectrum flux, zero-crossing, Entropy and MelFrequency Cepstral Coefficients (MFCCs). Sequential Forward Selection (SFS) was used to find out most suitable acoustic features from the computed feature set. This study used K-Nearest Neighbors (K-NN) classifier was used to detect the pronunciation mistakes from Arabic phonemes. This research selected the set of most discriminative acoustic features for each phoneme. K-NN achieved accuracy of 92.15% for mispronunciation detection of Arabic Phonemes

Interpreting acoustic features for the assessment of Alzheimer’s disease using ForestNet

Nowadays, interpretable machine learning models are one of the most critical topics in the medical domain. The lack of interpretation leads to blind and unreliable models for clinicians, despite the fact that the aim is to support diagnosis through these tools. This problem has been increasing since the creation of large models such as those based on deep learning, which, despite providing good performance in prediction and classification tasks, are not transparent to human understanding. One of the increasingly prevalent clinical problems related to acoustic and linguistic disorders is Alzheimer’s disease (AD), where one important challenge is to provide speech markers that help in supporting, understanding, and facilitating the diagnosis and monitoring of the disease. It motivates this study which proposes a methodology focused on analyzing acoustic features in AD and at the same time providing interpretation from the results. The proposed approach consists of using decision tree-based methods together with neural networks (ForestNet) for analyzing the classification results. Only features that can give interpretation were considered. Unweighted average recalls of up to 79% were achieved for discriminating AD patients. Then, we looked at the relevant features that provided most of the information for assessing AD, which were those related to rhythm, voiced rates, duration, and phone rates. This confirms that this kind of approach can be suitable for the discrimination of AD while maintaining a good performance. • An approach based on decision trees together with artificial neural networks (ForesNet) is considered for analyzing/interpreting different sets of acoustic features for the assessment of Alzheimer’s disease. • Unweighted average recalls of up to 79% were achieved for discriminating Alzheimer’s patients. • The proposed approach outperformed the use of regular decision tree-based methods by 11%. • The considered methods and results here may be interpretative from the clinical point of view, which makes our model suitable for supporting the clinical diagnosis of Alzheimer’s disease.

Acoustic Structural Integrity Assessment of Ceramics using Supervised Machine Learning and Uncertainty-Based Rejection

Industry and Quality 4.0 pose the opportunity to integrate artificial intelligence-based technology into the quality management of products/services. Particularly, quality control procedures of tableware ceramics require a demanding and faulty human manual (visual and acoustic) inspection. In this paper, we propose an uncertainty-aware automated acoustic inspection using a supervised machine learning model based on a set of novel acoustic features to classify ceramic plates, as cracked and uncracked. We conducted experiments on a dataset of 31 ceramic plates (16 cracked and 15 uncracked), collected in the laboratory. Data quality check and augmentation strategies were also performed, resulting in 2900 samples. The main contributions of this paper are: 1) description of 192 features selected for the acoustic inspection of ceramic plates; 2) comparison of model calibration results regarding three different classifiers; 3) study of different sources of uncertainty for classification with rejection option, through uncertainty quantification measures, and the effect of feature selection on it. We performed two experiments that differ in the usage of a supervised feature selection method. We split the augmented dataset into train/test sets in a proportion of 90/10. The calibrated SVM was selected as the best classifier based on model calibration and cross-validation results and was used in the prediction on the test set. The uncertainty-based rejection improved the train and test sets' classification results. In the experiment with feature selection, the classification performance remained high, while the uncertainty about the predictions and the percentage of rejected samples decreased.