Unvoiced Segments Research Articles

The effect of detoxification on acoustic features of Mandarin speech in male heroin users.

This study aims to investigate the effect of detoxification on acoustic features of Mandarin speech. Speech recordings were collected from 66 male abstinent heroin users with different durations of drug detoxification, specifically early abstinent users with a detoxification duration of less than 2 years, sustained abstinent users with 2 years of detoxification, and long-term abstinent users with a detoxification duration of more than 2 years. The results of the acoustic analyses showed that early abstinent users exhibited lower loudness, relative energies of F1, F2, and F3, higher H1-A3, and fewer loudness peaks per second, as well as a longer average duration of unvoiced segments, compared to the sustained and long-term abstinent users. The findings suggest that detoxification may lead to a rehabilitation process in the speech production of abstinent heroin users (e.g., less vocal hoarseness). This study not only provides valuable insights into the effect of detoxification on speech production but also provides a theoretical basis for the speech rehabilitation and detoxification treatment of heroin users.

Acoustic Characteristics of Mandarin Speech in Male Drug Users

Drug use/addiction has a profound impact on the physical and mental health of individuals. Previous studies have indicated that drug users may experience speech perception disorders, including speech illusion and challenges in recognizing emotional speech. However, the influence of drugs on speech production, as another crucial aspect of speech communication, has not been thoroughly examined. Therefore, the current study aimed to investigate how drugs affect the acoustic characteristics of speech in Chinese male drug users. Speech recordings were collected from a total of 160 male drug users (including 106 heroin users, 23 ketamine users, and 31 methamphetamine users) and 55 male healthy controls with no history of drug use. Acoustic analysis was conducted on the collected speech data from these groups, and classification analysis was performed using five supervised learning algorithms. The results demonstrated that drug users exhibited smaller F0 standard deviation, reduced loudness, cepstral peak prominence, and formant relative energies, as well as higher H1-A3, longer unvoiced segments, and fewer voiced segments per second compared to the control group. The classification analyses yielded good performance in classifying drug users and non-drug users, with an accuracy above 86%. Moreover, the identification of the three groups of drug users achieved an accuracy of approximately 70%. Additionally, the study revealed different effects on speech production among the three types of drugs. The above findings indicate the presence of speech disorders, such as vocal hoarseness, in drug users, thus confirming the assumption that the acoustic characteristics of speech in drug users deviates from the norm. This study not only fills the knowledge gap regarding the effects of drugs on the speech production of Chinese male drug users but also provides a more comprehensive understanding of how drugs impact human behaviors. Furthermore, this research provides theoretical foundations of detoxification and speech rehabilitation for drug users.

U-Shaped Low-Complexity Type-2 Fuzzy LSTM Neural Network for Speech Enhancement

Speech enhancement (SE) aims to improve the intelligibility and perceptual quality of speech contaminated by noise signals through spectral or temporal changes. Deep learning models achieve speech enhancement and estimate the magnitude spectrum. This paper proposes a novel and computationally efficient deep learning model to enhance noisy speech. The model pre-processes the noisy speech magnitude by redistributing energy from high-energy voiced segments to low-energy unvoiced segments using an adaptive power law transformation while maintaining the total energy of the speech signals constant. A U-shaped fuzzy long short-term memory (UFLSTM) estimates the magnitude of a time-frequency (T-F) mask by using the pre-processed data. Residual connections to the similar-shaped layers are added to avoid gradient decay. Attention process is adopted by modifying the forget gate of UFLSTM. To make a causal speech enhancement system, the processing does not include any future audio frames. We compare the proposed speech enhancement to other deep learning models in different noisy environments with signal-to-noise ratios of 0 dB, 5 dB, and 10 dB. The experiments show that the proposed SE system outscores the competing deep learning models and considerably improves speech intelligibility and quality. In terms of STOI and PESQ, the LibriSpeech database improves results by (0.211) 21.1% and (0.95) 36.39%, respectively, over noisy speech in seen noisy conditions, and by (0.199) 19.9% and (0.94) 35.69% over noisy speech in unseen noisy conditions. Further, the cross-corpus analysis shows that proposed SE system performs better when trained with the DNS dataset as compared to the LibriSpeech, VoiceBank, and TIMIT datasets.

An Electroglottograph Auxiliary Neural Network for Target Speaker Extraction

The extraction of a target speaker from mixtures of different speakers has attracted extensive amounts of attention and research. Previous studies have proposed several methods, such as SpeakerBeam, to tackle this speech extraction problem using clean speech from the target speaker to provide information. However, clean speech cannot be obtained immediately in most cases. In this study, we addressed this problem by extracting features from the electroglottographs (EGGs) of target speakers. An EGG is a laryngeal function detection technology that can detect the impedance and condition of vocal cords. Since EGGs have excellent anti-noise performance due to the collection method, they can be obtained in rather noisy environments. In order to obtain clean speech from target speakers out of the mixtures of different speakers, we utilized deep learning methods and used EGG signals as additional information to extract target speaker. In this way, we could extract target speaker from mixtures of different speakers without needing clean speech from the target speakers. According to the characteristics of the EGG signals, we developed an EGG_auxiliary network to train a speaker extraction model under the assumption that EGG signals carry information about speech signals. Additionally, we took the correlations between EGGs and speech signals in silent and unvoiced segments into consideration to develop a new network involving EGG preprocessing. We achieved improvements in the scale invariant signal-to-distortion ratio improvement (SISDRi) of 0.89 dB on the Chinese Dual-Mode Emotional Speech Database (CDESD) and 1.41 dB on the EMO-DB dataset. In addition, our methods solved the problem of poor performance with target speakers of the same gender and the different between the same gender situation and the problem of greatly reduced precision under the low SNR circumstances.

Significance of voiced and unvoiced speech segments for the detection of common cold.

This work investigates the significance of the voiced and unvoiced region for detecting common cold from the speech signal. In literature, the entire speech signal is processed to detect the common cold and other diseases. This study uses a short-time energy-based approach to segment the voiced and unvoiced region of the speech signal. Then, frame-wise mel frequency cepstral coefficients (MFCC) features are extracted from the voiced and unvoiced segments of each speech utterance, and statistics (mean, variance, skewness, and kurtosis) are calculated to get the feature vector for each speech utterance. The support vector machine (SVM) is utilized to analyze the performance of features extracted from the voiced and unvoiced region. Result shows that the feature extracted from voiced segments, unvoiced segments, and complete active speech (CAS) gives almost similar results using the MFCC features and SVM classifier. Therefore, rather than processing the CAS, we can process the unvoiced speech segments, which have fewer frames compared to CAS and voiced regions of speech. The processing of solely unvoiced segments can reduce the time and computation complexity of a speech signal-based common cold detection system.

Investigating the role of harmonic cancellation in speech-on-speech masking

This study investigated the role of harmonic cancellation in the intelligibility of speech in “cocktail party” situations. While there is evidence that harmonic cancellation plays a role in the segregation of simple harmonic sounds based on fundamental frequency (F0), its utility for mixtures of speech containing non-stationary F0s and unvoiced segments is unclear. Here we focused on the energetic masking of speech targets caused by competing speech maskers. Speech reception thresholds were measured using seven maskers: speech-shaped noise, monotonized and intonated harmonic complexes, monotonized speech, noise-vocoded speech, reversed speech and natural speech. These maskers enabled an estimate of how the masking potential of speech is influenced by harmonic structure, amplitude modulation and variations in F0 over time. Measured speech reception thresholds were compared to the predictions of two computational models, with and without a harmonic cancellation component. Overall, the results suggest a minor role of harmonic cancellation in reducing energetic masking in speech mixtures.

Influence of Alignment Strategies and Consonant Acoustic Features on Laterality Index in Dichotic Testing in Indian Languages

The right ear advantage (REA) observed for dichotic CV (DCV) testing in Indian languages is seen to be smaller compared with the observations in several studies in English and other languages. The present study aims to explore whether the differences in the temporal alignment of consonant-vowel (CV) stimuli used for the dichotic task is a contributing factor that can explain the smaller REAs observed in Indian languages. The study compared the laterality index (LI) values between DCV test results obtained using CV stimuli that were temporally aligned at the acoustic onset versus the burst onset, with 0 milliseconds lag between the right and left ears. A total of 50 right-handed individuals (native speakers of Kannada language) in the age range of 17 to 30 years with normal hearing sensitivity participated in the study. The participants' responses to the two sets of DCV stimuli were recorded and compared. LI was calculated for the overall scores as well as for specific voiced and unvoiced segment combinations. To test the reliability of the findings, 10 of the participants were re-tested using the same stimuli and the data were checked for reliability. The overall LI was 15.1% for the stimuli temporally aligned at the burst onset, whereas it was 5.7% for the stimuli temporally aligned at the acoustic onset. The difference in LI was not significant between the alignment conditions for unvoiced-unvoiced and voiced-voiced CV combinations, but this difference was significant for the unvoiced-voiced and voiced-unvoiced CV combinations. Comparing test and re-test scores showed good reliability. The results support our hypothesis that smaller REA observed in DCV task in Indian languages was due to the temporal alignment of CV stimuli. In the acoustic-onset-aligned condition, presenting pairs contrasting in the voicing feature tends to cancel out a relative ear advantage.

Digital Audio Forensics: Microphone and Environment Classification Using Deep Learning

The recording device along with the acoustic environment plays a major role in digital audio forensics. We propose an acoustic source identification system in this paper, which includes identifying both the recording device and the environment in which it was recorded. A hybrid Convolutional Neural Network (CNN) with Long Short-Term Memory (LSTM) is used in this study to automatically extract environments and microphone features from the speech sound. In the experiments, we investigated the effect of using the voiced and unvoiced segments of speech on the accuracy of the environment and microphone classification. We also studied the effect of background noise on microphone classification in 3 different environments, i.e., very quiet, quiet, and noisy. The proposed system utilizes a subset of the KSU-DB corpus containing 3 environments, 4 classes of recording devices, 136 speakers (68 males and 68 females), and 3600 recordings of words, sentences, and continuous speech. This research combines the advantages of both CNN and RNN (in particular bidirectional LSTM) models, called CRNN. The speech signals were represented as a spectrogram and were fed to the CRNN model as 2D images. The proposed method achieved accuracies of 98% and 98.57% for environment and microphone classification, respectively, using unvoiced speech segments.

A Deep Learning Based Method for Parkinson’s Disease Detection Using Dynamic Features of Speech

Detection of voice changes in Parkinson’s Disease (PD) patients would make it possible for early detection and intervention before the onset of disabling physical symptoms. This study explores static and dynamic speech features relating to PD detection. A comparative analysis of the articulation transition characteristics shows that the number of articulation transitions and the trend of the fundamental frequency curve are significantly different between HC speakers and PD patients. Motivated by this observation, we propose to apply Bidirectional long-short term memory (LSTM) model to capture time-series dynamic features of a speech signal for detecting PD. The dynamic speech features are measured based on computing the energy content in the transition from unvoiced to voiced segments (onset), and in the transition from voiced to unvoiced segments (offset). Under the two evaluation methods of 10-fold cross validation (CV) and splitting the dataset without samples overlap of one individual, the experimental results show that the proposed method remarkably improves the accuracy of PD detection over traditional machine learning models using static features.

BioVoice: A multipurpose tool for voice analysis

ObjectiveThis paper introduces BioVoice, a user-friendly software tool for the acoustical analysis of the human voice. It estimates more than 20 acoustical parameters with advanced and robust analysis techniques specifically developed for different vocal emissions, from the newborn to the adult and the singer. MethodsBioVoice performs both time and frequency analyses, detecting the number, length, and percentage of voiced and unvoiced segments and computing fundamental frequency (F0), formant frequencies (F1-F3), noise level, and jitter. The software tool computes the melodic shape of F0 out of 12 basic shapes and allows performing perceptual analysis in newborn and child voice. In the singing voice case, formants up to F5 are computed as well as the quality ratio and parameters concerning vibrato and its regularity. Colour figures and Excel tables show F0, the spectrogram with formants, voiced segments, and quality ratio. ResultsExamples of voice analysis in adults, children, newborns, and singers are presented. They show the specific capabilities and the high performance of BioVoice also as compared to another existing software tool. SignificanceBioVoice is a free user-friendly software tool for voice analysis that implements new estimation techniques. Basic parameters are computed as well as new ones specifically developed for newborn cry and singing voice analysis, not available with current software tools. ConclusionsBioVoice is capable to deal with low to high pitched voices implementing dedicated tools. Thanks to its simple and intuitive interface, colour figures and Excel tables, it is a valuable tool suitable also for the inexperienced user.

Effect of Voiced Segment Extraction on Voice of Vocal Cord Paralysis: Focused on Cepstral Analysis

Background and Objectives The purpose of this study was to investigate the effect of unvoiced segments on the cepstral analysis in patients with vocal cord paralysis (VCP).Subjects and Method A total 302 subjects (173 subjects with VCP and 129 normal voice subjects) participated in this study. The sustained vowel /a/ 2 seconds and one sentence of ‘Sanchaek’ were edited, and analyzed by Praat script. The cepstral analyses were performed using sustained vowel (SV), continuous speech (CS), and extracted continuous speech (EXT) samples. The auditory-perceptual (AP) rating was also completed by three raters.Results First, there were significant differences in all variables except low-to high spectral ratio(L/H ratio)_EXT between two groups. Second, cepstral peak prominence (CPP), smoothed cepstral peak prominence (CPPS), and L/H ratio showed significant differences in SV, CS, and EXT samples. Third, cepstral measurements were highly correlated with the AP ratings. Finally, the level of discrimination of dysphonia estimated from CPP and CPPS gotten from SV and CS values was more than area under the curve (AUC) of 0.941. AUC of 0.880 or more was also found in EXT.Conclusion In this study, we confirmed that both CS and EXT are highly predictive of pathologic speeches. Further study will also need to be validated for a more diverse group of voice disorders.

Characteristics of the Voice of Dysphonic School Children from 4:0 to 7:11 Years Old

Childhood dysphonia is frequent and has many origins, with the most common initial symptom being the altered vocal quality. Describe and correlate the auditory-perceptual and acoustic characteristics of the voice, and the maximum phonation time (MPT), of dysphonic schoolchildren from private and public schools from 4:0 to 7:11 years old, female and male. MPT, acoustic (Multi-Dimensional Voice Program Advanced and Real Time Spectrogram), and auditory-perceptual (Consensus Auditory-Perceptual Evaluation of Voice) evaluations of 115 schoolchildren were statistically analyzed. Auditory-perceptual parameters values were classified as mild, and MPT values were reduced, in most children. Fundamental frequency and Jitt values decreased significantly with age; MPT/i/ increased significantly with age. Relative average perturbation, voice turbulence index (VTI), and number of sub-harmonic segments values were significantly higher in 4:0 years old children. The number of unvoiced segments was higher in 6:0 years old children. Shimmer percent was higher in 7:0 years old children. There were negative correlations between the high frequency spectrographic tracing color intensity and the vocal strain; the VTI and the MPT; the degree of unvoiced segments and the MPT. There was a positive correlation between the smoothed pitch period perturbation quotient and the roughness. In dysphonic schoolchildren, auditory-perceptual parameters were moderate, MPT was reduced, several acoustic parameters were altered, and these vocal characteristics improved with age.

Monaural speech separation using GA-DNN integration scheme

In this research work, we propose the model based on the Genetic Algorithm (GA) and Deep Neural Network (DNN) to enhance the quality and intelligibility of the noisy speech. In this proposed model, the Voiced Speech (VS) T-F mask is computed using correlogram, frame energy and cross-channel correlogram and Unvoiced Speech (UVS) T-F mask is computed using speech onset/offset. The T-F mask obtained using speech onset and offset represents both voiced and unvoiced segment of the noisy speech signal. The UVS T-F mask is obtained by subtracting the VS from the T-F mask obtained earlier using speech onset/offset. Next, the GA is used to find the optimum weight to combine the T-F mask of VS and UVS to improve speech quality and intelligibility. The weight obtained using GA may not be an optimum one for all sets of speech and noise. This research work focuses on this issue and proposes a DNN model to estimate the optimum weight for all sets of speech and noise. The DNN model is trained using features and optimum weight obtained using GA. Later, the trained DNN model is used to estimate the optimum weight for the testing speech and noise samples. The performance of the proposed GA-DNN based model is evaluated using objective and subjective quality and intelligibility measures. The results of the proposed model shows a prompt improvement in the speech quality and intelligibility with average of 0.73, 4.07, 0.17, 0.26 and 0.22 for PESQ, SNR, STOI, CSII and NCM when compared with the existing speech separation systems.

A continuous vocoder for statistical parametric speech synthesis and its evaluation using an audio-visual phonetically annotated Arabic corpus

In this paper, we present an extension of a novel continuous residual-based vocoder for statistical parametric speech synthesis by addressing two objectives. First, because the noise component is often not accurately modelled in modern vocoders (e.g. STRAIGHT), a new technique for modelling unvoiced sounds is proposed by adding time domain envelope to the unvoiced segments to avoid any residual buzziness. Four time-domain envelopes (Amplitude, Hilbert, Triangular and True) are investigated, enhanced, and then applied to the noise component of the excitation in our continuous vocoder, i.e. of which all parameters are continuous. With the future aim of producing high-quality Arabic speech synthesis, we secondly apply this vocoder on a modern standard Arabic audio-visual corpus which is annotated both phonetically and visually, and dedicated to emotional speech processing studies.In an objective experiment, we investigated the Phase Distortion Deviation, whereas a MUSHRA type subjective listening test was conducted comparing natural and vocoded speech samples. As a result, both experiments based on the proposed noise modelling have shown satisfactory results in terms of naturalness and intelligibility, while outperforming STRAIGHT and other earlier residual-based approaches.

Continuous vocoder applied in deep neural network based voice conversion

In this paper, a novel vocoder is proposed for a Statistical Voice Conversion (SVC) framework using deep neural network, where multiple features from the speech of two speakers (source and target) are converted acoustically. Traditional conversion methods focus on the prosodic feature represented by the discontinuous fundamental frequency (F0) and the spectral envelope. Studies have shown that speech analysis/synthesis solutions play an important role in the overall quality of the converted voice. Recently, we have proposed a new continuous vocoder, originally for statistical parametric speech synthesis, in which all parameters are continuous. Therefore, this work introduces a new method by using a continuous F0 (contF0) in SVC to avoid alignment errors that may happen in voiced and unvoiced segments and can degrade the converted speech. Our contribution includes the following. (1) We integrate into the SVC framework the continuous vocoder, which provides an advanced model of the excitation signal, by converting its contF0, maximum voiced frequency, and spectral features. (2) We show that the feed-forward deep neural network (FF-DNN) using our vocoder yields high quality conversion. (3) We apply a geometric approach to spectral subtraction (GA-SS) in the final stage of the proposed framework, to improve the signal-to-noise ratio of the converted speech. Our experimental results, using two male and one female speakers, have shown that the resulting converted speech with the proposed SVC technique is similar to the target speaker and gives state-of-the-art performance as measured by objective evaluation and subjective listening tests.

Glottal Activity Detection from the Speech Signal Using Multifractal Analysis

This work proposes a novel method for the detection of glottal activity regions from the speech signal. Glottal activity detection refers to the problem of discriminating voiced and unvoiced segments of the speech signal. This is a fundamental step in the work flow of many speech processing applications. Much of the existing approaches for voiced/unvoiced detection are based on linear measures though the speech is produced from an underlying nonlinear process. The present work solves the problem from a nonlinear perspective, using the framework of multifractal analysis. The fractal property of the speech signal during the production of voiced and unvoiced sounds is sought to obtain the characterization of glottal activity. The characterization is done by computing the Hurst exponent from the evaluation of the scaling property of fluctuations present in the speech signal. Experimental analysis shows that Hurst exponent varies consistently with respect to the dynamics of glottal activity. The performance of the proposed method has been evaluated on the CMU-arctic, Keele and KED-Timit databases with simultaneous electroglottogram signals. Experimental results show that the average detection accuracy or error rate of the proposed method is comparable to the best performing algorithm on clean speech signals. Besides, evaluation of the robustness of the proposed method to noise degradation shows comparable results with other methods for signal-to-noise ratio greater than 10 dB and 20 dB, respectively, for white noise and babble noise.

Automatic speech analysis to early detect functional cognitive decline in elderly population.

This study aimed at evaluating whether people with a normal cognitive function can be discriminated from subjects with a mild impairment of cognitive function based on a set of acoustic features derived from spontaneous speech. Voice recordings from 90 Italian subjects (age >65 years; group 1: 47 subjects with MMSE>26; group 2: 43 subjects with 20≤ MMSE ≤26) were collected. Voice samples were processed using a MATLAB-based custom software to derive a broad set of known acoustic features. Linear mixed model analyses were performed to select the features able to significantly distinguish between groups. The selected features (% of unvoiced segments, duration of unvoiced segments, % of voice breaks, speech rate, and duration of syllables), alone or in addition to age and years of education, were used to build a learning-based classifier. The leave-one-out cross validation was used for testing and the classifier accuracy was computed. When the voice features were used alone, an overall classification accuracy of 0.73 was achieved. When age and years of education were additionally used, the overall accuracy increased up to 0.80. These performances were lower than the accuracy of 0.86 found in a recent study. However, in that study the classification was based on several tasks, including more cognitive demanding tasks. Our results are encouraging because acoustic features, derived for the first time only from an ecologic continuous speech task, were able to discriminate people with a normal cognitive function from people with a mild cognitive decline. This study poses the basis for the development of a mobile application performing automatic voice analysis on-the-fly during phone calls, which might potentially support the detection of early signs of functional cognitive decline.

노인성 음성장애에 대한 켑스트럼 분석에서의 무성음 분절 영향

노인성 음성장애에 대한 켑스트럼 분석에서의 무성음 분절 영향

Improving Speech Intelligibility in Monaural Segregation System by Fusing Voiced and Unvoiced Speech Segments

Improving the speech intelligibility remains a challenging problem in digital hearing aids. This research work proposes a new speech segregation algorithm to improve the speech intelligibility by effectively fusing the voiced and unvoiced segment of the speech signal using the genetic algorithm. The voiced speech segments are obtained using perceptual speech cues such as auto-correlation, cross-channel correlation and pitch. Similarly, the unvoiced speech segments are obtained using another perceptual speech cue onset/offset after subtracting the voiced segments. The speech onset- and offset-based segregation process actually produce segments for both voiced and unvoiced. The unvoiced speech segments are obtained by subtracting the voiced speech segments from the segments obtained using speech onset and offset. The unvoiced speech segments obtained using onset and offset may contain interference. This research work proposes a scheme to remove those interferences from the unvoiced speech segments and effectively fuse the segments of voiced and unvoiced speech using the genetic algorithm. The performance of the proposed algorithm is evaluated using the intelligibility measures such as CSII, NCM and STOI. The experimental results show that the proposed algorithm significantly improves the speech intelligibility with an average of 0.23 for CSII, 0.20 for NCM and 0.16 for STOI as compared with other existing systems.

Intertext Variability of Smoothed Cepstral Peak Prominence, Methods to Control It, and Its Diagnostic Properties

This study aimed to estimate the intertext variability of smoothed cepstral peak prominence (CPPS), examine whether sound-processing techniques improved its variability and diagnostic capability, and evaluate the degree of intertext variability in detail with reference to the CPPS variabilities in sustained vowels. This was a retrospective study. Text readings of 58 Japanese syllables were recorded from 210 speakers with different diagnoses and varying degrees of dysphonia, and were divided into six passages. Applying the sound-processing techniques to those passages, we prepared three sample types: (1) nonprocessed, (2) only-loud, and (3) only-voiced samples. The intertext CPPS variability and diagnostic properties were compared across the passages and sample types. For detailed analysis, we subsequently extracted 63 normophonic speakers who maintained constant quality in their vowel utterances to evaluate the degree of intertext CPPS variability in relation to the variabilities between repeated identical vowels and across different vowels. Although several combinations of passages showed moderate-to-large CPPS variabilities, those variabilities were decreased by either technique, especially the deletion of silent segments, which resulted in the best diagnostic accuracy. The degree of intertext CPPS variability for the only-voiced samples was comparable to that of the CPPS variabilities in sustained vowels. The sound-processing technique removing silent segments should be applied to enhance the diagnostic properties of CPPS. The additional technique of deleting unvoiced segments is worth adopting if clinicians and researchers seek to attenuate the influence of text differences in calculating CPPS values.