Throat Microphone Research Articles

Enhancement of throat microphone speech using Empirical Mode Decomposition (EMD)

Enhancement of throat microphone speech using Empirical Mode Decomposition (EMD)

Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone and throat microphones

Speech is produced by a nonlinear, dynamical Vocal Tract (VT) system, and is transmitted through multiple (air, bone and skin conduction) modes, as captured by the air, bone and throat microphones respectively. Speaker specific characteristics that capture this nonlinearity are rarely used as stand-alone features for speaker modeling, and at best have been used in tandem with well known linear spectral features to produce tangible results. This paper proposes Recurrent Plot (RP) embeddings as stand-alone, non-linear speaker-discriminating features. Two datasets, the continuous multimodal TIMIT speech corpus and the consonant-vowel unimodal syllable dataset, are used in this study for conducting closed-set speaker identification experiments. Experiments with unimodal speaker recognition systems show that RP embeddings capture the nonlinear dynamics of the VT system which are unique to every speaker, in all the modes of speech. The Air (A), Bone (B) and Throat (T) microphone systems, trained purely on RP embeddings perform with an accuracy of 95.81%, 98.18% and 99.74%, respectively. Experiments using the joint feature space of combined RP embeddings for bimodal (A–T, A–B, B–T) and trimodal (A–B–T) systems show that the best trimodal system (99.84% accuracy) performs on par with trimodal systems using spectrogram (99.45%) and MFCC (99.98%). The 98.84% performance of the B–T bimodal system shows the efficacy of a speaker recognition system based entirely on alternate (bone and throat) speech, in the absence of the standard (air) speech. The results underscore the significance of the RP embedding, as a nonlinear feature representation of the dynamical VT system that can act independently for speaker recognition. It is envisaged that speech recognition too will benefit from this nonlinear feature.

Assessment of radio(chemo)therapy-related dysphagia in head and neck cancer patients based on cough-related acoustic features: a prospective phase II national clinical trial (ACCOUGH-P/A trial)

BackgroundRadiation-associated dysphagia is defined as impaired swallowing efficiency/safety following (chemo)radiotherapy in head and neck cancer patients. In a dysphagia framework, impaired coughing may lead to lung aspiration and fatal lung infection. Although cough efficacy is a predictor of the risk of aspiration, cough investigation is minimal in patients with radiation-associated dysphagia. Because cough is a transient signal, existing software for speech analysis are not appropriate. The goal of our project is to develop an assessment method using acoustic features related to voluntary and reflexive coughs as biomarkers of the risk of penetration/aspiration in patients with radiation-associated dysphagia.MethodsHealthy subjects and head and neck cancer patients with and without dysphagia will produce voluntary coughs, throat clearings and reflexive coughs. Recordings will be made using an acoustic microphone and a throat microphone. The recorded signals will be manually segmented and subsequently analysed with a software under development. Automatic final segmentation enables to measure cough duration. The first method of analysis includes temporal features: the amplitude contour, the sample entropy and the kurtosis. These features report respectively the strength, the unpredictability (turbulence noise due to the air jet) and the impulsive quality (burst) of the signal. The second method of analysis consists of a spectral decomposition of the relative cough signal energy into several frequency bands (0–400 Hz, 400–800 Hz, 800–1600 Hz, 1600–3200 Hz, > 3200 Hz). The primary outcome of this exploratory research project is the identification of a set of descriptive acoustic cough features in healthy subjects as reference data (ACCOUGH). The secondary outcome of this research in head and neck cancer patients with radiation-associated dysphagia includes the identification of (1) a set of descriptive acoustic cough features as biomarkers of penetration-aspiration (ACCOUGH-P/A), (2) swallowing scores, (3) voice features and (4) aerodynamic cough features.DiscussionThis study is expected to develop methods of acoustic cough analysis to enhance the assessment of radiation-associated dysphagia in head and neck cancer patients following (chemo)radiation.Trial registrationInternational Standard Randomized Controlled Trials Number (ISRCTN) registry ISRCTN16540497. Accepted on 23 June 2023.

Effects of microphone mounting location and gender on accuracy in speech recognition using a throat microphone.

Speech recognition using air-conduction microphones is less accurate under high noise conditions and when the volume of the speaker's voice is relatively low. In this study, the effect of mounting location of throat microphones (which are less susceptible to ambient noise) on recognition accuracy was experimentally investigated. The results confirmed that mounting position and speaker gender affected recognition accuracy, regardless of any other factor or speech recognition system. In addition, relatively lower recognition accuracy was observed in the upper part of the neck near the mandibular angle for both males and females.

A Study on the Impact of Lombard Effect on Recognition of Hindi Syllabic Units Using CNN Based Multimodal ASR Systems

Research work on the design of robust multimodal speech recognition systems making use of acoustic and visual cues, extracted using the relatively noise robust alternate speech sensors is gaining interest in recent times among the speech processing research fraternity. The primary objective of this work is to study the exclusive influence of Lombard effect on the automatic recognition of the confusable syllabic consonant-vowel units of Hindi language, as a step towards building robust multimodal ASR systems in adverse environments in the context of Indian languages which are syllabic in nature. The dataset for this work comprises the confusable 145 consonant-vowel (CV) syllabic units of Hindi language recorded simultaneously using three modalities that capture the acoustic and visual speech cues, namely normal acoustic microphone (NM), throat microphone (TM) and a camera that captures the associated lip movements. The Lombard effect is induced by feeding crowd noise into the speaker’s headphone while recording. Convolutional Neural Network (CNN) models are built to categorise the CV units based on their place of articulation (POA), manner of articulation (MOA), and vowels (under clean and Lombard conditions). For validation purpose, corresponding Hidden Markov Models (HMM) are also built and tested. Unimodal Automatic Speech Recognition (ASR) systems built using each of the three speech cues from Lombard speech show a loss in recognition of MOA and vowels while POA gets a boost in all the systems due to Lombard effect. Combining the three complimentary speech cues to build bimodal and trimodal ASR systems shows that the recognition loss due to Lombard effect for MOA and vowels reduces compared to the unimodal systems, while the POA recognition is still better due to Lombard effect. A bimodal system is proposed using only alternate acoustic and visual cues which gives a better discrimination of the place and manner of articulation than even standard ASR system. Among the multimodal ASR systems studied, the proposed trimodal system based on Lombard speech gives the best recognition accuracy of 98%, 95%, and 76% for the vowels, MOA and POA, respectively, with an average improvement of 36% over the unimodal ASR systems and 9% improvement over the bimodal ASR systems.

Improving communications in PPE: a solution for ‘landline’ telephone communication

BackgroundEmergency care staff wearing elastomeric respiratory personal protective equipment (PPE) report difficulties in communicating by telephone. We developed and tested an affordable technological solution aimed at improving telephone call intelligibility...

Sound Features of Oropharyngeal Activity Sound by Using Throat Microphone

Sound Features of Oropharyngeal Activity Sound by Using Throat Microphone

Semi-sparse Residual Recurrent Neural Network via dictionary representation for throat microphone quality enhancement

Throat microphone (TM) speech can be used for communication in noisy environment as it collects signals directly from human skin, but it should be improved in clarity and intelligibility due to the severe loss of high-frequency components. As recovery directly by neural networks is not sufficient to achieve satisfactory performance, we propose a dictionary representation based neural network to address this issue. Specifically, a magnitude spectrum dictionary of air-conducted speech is computed via sparse non-negative matrix factorization (SNMF), and then it is used to represent the transformed speech in hidden layer of the network. Meanwhile, a compensating dictionary is adopted to improve the representation accuracy. A memory efficient Semi-sparse Residual Recurrent Neural Network (SResRNN) with interactive mechanism and a special ResNet is employed to generate the coefficients on the dictionaries. Lastly, a three-layer neural network using a special initialization scheme is constructed as the recovery model. In the experiments, the model is compared with other five recovering models, and different criteria are adopted to measure the performance, the objective and subjective results can demonstrate the superiority of our proposed model.

Noise Reduction Combining a General Microphone and a Throat Microphone.

In this study, we propose a method to reduce noise from speech obtained from a general microphone using the information of a throat microphone. A throat microphone records a sound by detecting the vibration of the skin surface near the throat directly. Therefore, throat microphones are less prone to noise than ordinary microphones. However, as the acoustic characteristics of the throat microphone differ from those of ordinary microphones, its sound quality degrades. To solve this problem, this study aims to improve the speech quality while suppressing the noise of a general microphone by using the information recorded by a throat microphone as reference information to extract the speech signal in general microphones. In this paper, the framework of the proposed method is formulated, and several experiments are conducted to evaluate the noise suppression and speech quality improvement effects of the proposed method.

A multimodal Lombard speech recognition system for the confusable Hindi syllabic units

Research work on the design of robust multimodal speech recognition systems making use of acoustic and visual cues, extracted using the relatively noise robust alternate speech sensors is gaining interest in recent times among the speech processing research fraternity. The primary objective of this work is to study the exclusive influence of Lombard effect on Automatic Speech Recognition (ASR) systems towards building robust multimodal ASR systems in adverse environments in the context of Indian languages which are syllabic in nature. The dataset for this work comprises the confusable 145 Consonant-Vowel (CV) syllabic units of Hindi language recorded simultaneously using three modalities that capture the acoustic and visual speech cues, namely Normal acoustic Microphone (NM), Throat Microphone (TM) and a camera that captures the associated lip movements. The Lombard effect is induced by feeding crowd noise into the speaker’s headphone while recording. HMM models are built to categorize the CV units based on their Place of Articulation (POA), Manner Of Articulation (MOA) and vowels (under clean and Lombard conditions). Unimodal ASR systems built using each speech cue show a recognition loss in all the systems due to Lombard effect. To overcome this loss, the complimentary speech cues taken from normal and throat microphone Lombard speech as well as from visual Lombard speech are used to build three bimodal and one trimodal ASR systems. Among the ASR systems studied, the trimodal system gives the best recognition accuracy of 98%, 95% and 76% for the vowels, MOA and POA, respectively, with an average improvement of 36% over the unimodal ASR systems and 9% improvement over the bimodal ASR systems.

Speaker Recognition for Flexible Throat Microphone Via Supervised Contrastive Learning

Speaker Recognition for Flexible Throat Microphone Via Supervised Contrastive Learning

Improving phoneme recognition of throat microphone speech recordings using transfer learning

Throat microphones (TM) are a type of skin-attached non-acoustic sensors, which are robust to environmental noise but carry a lower signal bandwidth characterization than the traditional close-talk microphones (CM). Attaining high-performance phoneme recognition is a challenging task when the training data from a degrading channel, such as TM, is limited. In this paper, we address this challenge for the TM speech recordings using a transfer learning approach based on the stacked denoising auto-encoders (SDA). The proposed transfer learning approach defines an SDA-based domain adaptation framework to map the source domain CM representations and the target domain TM representations into a common latent space, where the mismatch across TM and CM is eliminated to better train an acoustic model and to improve the TM phoneme recognition. For the phoneme recognition task, we use the convolutional neural network (CNN) and the hidden Markov model (HMM) based CNN/HMM hybrid system, which delivers better acoustic modeling performance compared to the conventional Gaussian mixture model (GMM) based models. In the experimental evaluations, we observed more than 12% relative phoneme error rate (PER) improvement for the TM recordings with the proposed transfer learning approach compared to baseline performances.

Overcoming Communication Barriers: An evaluation of communication devices for healthcare providers wearing powered air-purifying respirators (PAPRs)

IntroductionThe COVID-19 pandemic has resulted in an increased use of Powered Air Purifying Respirators (PAPRs), by health care providers to mitigate the risk of viral transmission, especially for aerosol-generating procedures. In this study, we evaluate communication devices that could be used concurrently with PAPRs to promote improved communication. MethodsWe tested two devices, a Bluetooth earpiece and a throat microphone that operated over mobile networks, against a control scenario in a simulated operating room environment with participants donning PAPRs. Participants read a short paragraph to each other, transcribed short phrases, and evaluated the scenarios according to speech intelligibility, ease of use, and comfort. ResultsThere were 30 participants of varying PAPR experience. The Bluetooth headset had the most accurate transcriptions, followed by control, and lastly the neckpiece (94.7%vs 88.4%vs 76%, p<0.001). ConclusionCommunication devices have the potential to bridge but also worsen communications barriers between providers donning PAPRs.

Domain Adaptation for Food Intake Classification With Teacher/Student Learning

Automatic dietary monitoring (ADM) stands as a challenging application in wearable healthcare technologies. In this paper, we define an ADM to perform food intake classification (FIC) over throat microphone recordings. We investigate the use of transfer learning to design an improved FIC system. Although labeled data with acoustic close-talk microphones are abundant, throat data is scarce. Therefore, we propose a new adaptation framework based on teacher/student learning. The teacher network is trained over high-quality acoustic microphone recordings, whereas the student network distills deep feature extraction capacity of the teacher over a parallel dataset. Our approach allows us to transfer the representational capacity, adds robustness to the resulting model, and improves the FIC through throat microphone recordings. The classification problem is formulated as a spectra-temporal sequence recognition using the Convolutional LSTM (ConvLSTM) models. We evaluate the proposed approach using a large scale acoustic dataset collected from online recordings, an in-house food intake throat microphone dataset, and a parallel speech dataset. The bidirectional ConvLSTM network with the proposed domain adaptation approach consistently outperforms the SVM- and CNN-based baseline methods and attains 85.2% accuracy for the classification of 10 different food intake items. This translates to 17.8% accuracy improvement with the proposed domain adaptation.

Consequences and Solutions for the Impact of Communication Impairment on Noninvasive Ventilation Therapy for Acute Respiratory Failure: A Focused Review.

Objectives:With over 2 million cases of acute respiratory failure in the United States per year, noninvasive ventilation has become a leading treatment modality, often supplanting invasive mechanical ventilation as the initial treatment of choice. Most acute respiratory failure patients use a full face (oronasal) mask with noninvasive ventilation, which is known to impair communication, but its popularity and benefit has led many providers to accept the communication impairment. Medical staff periodically remove masks to communicate with patients, but patients are often limited to short utterances and risk lung derecruitment upon removal of positive pressure. These problems can lead to noninvasive ventilation failure, which is often linked to worse outcomes than first initiating invasive mechanical ventilation and can lead to increased hospitalization costs.Data Sources:We searched MEDLINE and Google Scholar for “speech,” “communication,” “impairment,” “failure,” “complications,” “NIPPV,” “NIV,” and “noninvasive ventilation.”Study Selection:We included articles with patients in acute respiratory failure. We excluded articles for patients using noninvasive ventilation therapy for obstructive sleep apnea.Data Synthesis:Communication impairment has been associated with increasing noninvasive ventilation anxiety (odds ratio, 1.25). Of patients using noninvasive ventilation, 48% require early discontinuation, 22% refuse noninvasive ventilation, and 9% are ultimately intubated. Improvements to communication have been shown to reduce fear and anxiety in invasive mechanical ventilation patients. Analogous communication problems exist with effective solutions in other fields, such as fighter pilot masks, that can be easily implemented to enhance noninvasive ventilation patient care, increase adherence to noninvasive ventilation treatment, and improve patient outcomes.Conclusions:Communication impairment is an underappreciated cause of noninvasive ventilation complications and failure and requires further characterization. Analogous solutions—such as throat microphones and mask-based microphones—that can be easily implemented show potential as cost-effective methods to reduce noninvasive ventilation failure.

Calling from the cockpit - [Past Forward

The open cockpit of a World War I biplane was not exactly conducive to easy conversation. Intense noise, vibration, and often-violent air disturbances drowned out the crew's voices. The muscles of the face had trouble retaining their shape. Yet the clear benefits of timely aerial reconnaissance stimulated wartime research into wireless telephony, to allow pilots to communicate with one another and the ground. During the summer of 1915, Charles Prince, an engineer and officer in the British Royal Flying Corps, successfully tested the first air-to-ground voice communication at Brooklands Aerodrome. Three years later, Prince's team unveiled the throat microphone [below], which when used with the aircraft's radio telephony transmitter allowed pilots to talk hands free, keeping their hands on the stick and their eyes on the sky.

Prolongation of Oral Phase for Initial Swallow of Solid Food is Associated with Oral Diadochokinesis Deterioration in Nursing Home Residents in Japan: A Cross-Sectional Study.

Prolongation of bolus forming complicates ingestion, in particular in older adults. The purpose of this study is to examine which oral functions are associated with prolongation of the oral phase of forming a bolus until swallowing in older adults. Cross-sectional study. three nursing homes in Kitakyushu, Japan from August 2017 to October 2018. 39 adults >60-years. Number of functional teeth, chewing ability, swallowing ability, tongue and cheek pressure, saliva flow rate, oral diadochokinesis, global cognitive function, and body mass index, were examined. Time of oral phase until the first swallowing of solid food was measured as the outcome of the study using video, and audio recording of the swallowing sound by a throat microphone, with the cutoff point designated at 30 s. Based on the oral phase, participants were divided in two groups: normal and prolonged. The 39 enrolled participants had a median age of 87 years, 17.3% were men, and 48.7% had prolonged oral phase. In the prolonged group, the swallowing ability, saliva flow rate, tongue and cheek pressure, and oral diadochokinesis were significantly lower than in the normal group. Binomial logistic regression analysis revealed that oral phase prolongation was associated with oral diadochokinesis (odds ratio 0.81, 95% confidence interval 0.67-0.98) after adjusting for potential covariates. Oral diadochokinesis deterioration is significantly associated with oral phase prolongation for initial swallowing of solid food in older adults.

PROLONGATION OF ORAL PHASE FOR INITIAL SWALLOW OF SOLID FOOD IS ASSOCIATED WITH ORAL DIADOCHOKINESIS DETERIORATION IN NURSING HOME RESIDENTS IN JAPAN: A CROSS-SECTIONAL STUDY

Background: Prolongation of bolus forming complicates ingestion, in particular in older adults. Objectives: The purpose of this study is to examine which oral functions are associated with prolongation of the oral phase of forming a bolus until swallowing in older adults. Design: Cross-sectional study. Setting: three nursing homes in Kitakyushu, Japan from August 2017 to October 2018. Participants: 39 adults &gt;60-years. Measurements: Number of functional teeth, chewing ability, swallowing ability, tongue and cheek pressure, saliva flow rate, oral diadochokinesis, global cognitive function, and body mass index, were examined. Time of oral phase until the first swallowing of solid food was measured as the outcome of the study using video, and audio recording of the swallowing sound by a throat microphone, with the cutoff point designated at 30 s. Based on the oral phase, participants were divided in two groups: normal and prolonged. Results: The 39 enrolled participants had a median age of 87 years, 17.3% were men, and 48.7% had prolonged oral phase. In the prolonged group, the swallowing ability, saliva flow rate, tongue and cheek pressure, and oral diadochokinesis were significantly lower than in the normal group. Binomial logistic regression analysis revealed that oral phase prolongation was associated with oral diadochokinesis (odds ratio 0.81, 95% confidence interval 0.67–0.98) after adjusting for potential covariates. Conclusion: Oral diadochokinesis deterioration is significantly associated with oral phase prolongation for initial swallowing of solid food in older adults.

A CNN based Speaker Recognition System using an Alternate Bone Microphone

State-of-art speaker recognition system uses acoustic microphone speech to identify/verify a speaker. The multimodal speaker recognition system includes modality of input data recorded using sources like acoustics mic,array mic ,throat mic, bone mic and video recorder. In this paper we implemented a multi-modal speaker identification system with three modality of speech as input, recorded from different microphones like air mic, throat mic and bone mic . we propose and claim an alternate way of recording the bone speech using a throat microphone and the results of a implemented speaker recognition using CNN and spectrogram is presented. The obtained results supports our claim to use the throat microphone as suitable mic to record the bone conducted speech and the accuracy of the speaker recognition system with signal speech recorded from air microphone get improved about 10% after including the other modality of speech like throat and bone speech along with the air conducted speech.

Joint Audio-Ultrasound Food Recognition for Noisy Environments.

Continuous recognition of ingested foods without user intervention is very useful for the pre-screening of obesity and diet-related disease. An automatic food recognition method that combines the two modalities of audio and ultrasonic signals (US) is proposed in this study. Under a noise-free environment, classification accuracy of an audio-only recognizer is generally higher than that of US-only recognizers, but the performance of US recognizers is unaffected by acoustic noise levels. In the recognition system presented herein, the likelihood score of the audio-US feature was given by a linear combination of class-conditional observation log-likelihoods for two classifiers, using the appropriate weights. We developed a weighting process adaptive to signal-to-noise ratios (SNRs). The main objective here involves determining the optimal SNR classification boundaries and constructing a set of optimum stream weights for each SNR class. A feasibility test was conducted to verify the usefulness of the proposed method by conducting recognition experiments on seven types of food. The performance was compared with conventional methods that use in-ear and throat microphones. The proposed method yielded remarkable levels of recognition performance of 90.13% for artificially added noise and 89.67% under actual noisy environments, when the SNR ranged from 0 to 20 dB.