Subjective Speech Quality Research Articles

Evaluation of digital watermarking on subjective speech quality

New methods of securing the distribution of audio content have been widely deployed in the last twenty years. Their impact on perceptive quality has, however, only been seldomly the subject of recent extensive research. We review digital speech watermarking state of the art and provide subjective testing of watermarked speech samples. Latest speech watermarking techniques are listed, with their specifics and potential for further development. Their current and possible applications are evaluated. Open-source software designed to embed watermarking patterns in audio files is used to produce a set of samples that satisfies the requirements of modern speech-quality subjective assessments. The patchwork algorithm that is coded in the application is mainly considered in this analysis. Different watermark robustness levels are used, which allow determining the threshold of detection to human listeners. The subjective listening tests are conducted following ITU-T P.800 Recommendation, which precisely defines the conditions and requirements for subjective testing. Further analysis tries to determine the effects of noise and various disturbances on watermarked speech’s perceived quality. A threshold of intelligibility is estimated to allow further openings on speech compression techniques with watermarking. The impact of language or social background is evaluated through an additional experiment involving two groups of listeners. Results show significant robustness of the watermarking implementation, retaining both a reasonable net subjective audio quality and security attributes, despite mild levels of distortion and noise. Extended experiments with Chinese listeners open the door to formulate a hypothesis on perception variations with geographical and social backgrounds.

Deep Gaussian process based multi-speaker speech synthesis with latent speaker representation

This paper proposes deep Gaussian process (DGP)-based frameworks for multi-speaker speech synthesis and speaker representation learning. A DGP has a deep architecture of Bayesian kernel regression, and it has been reported that DGP-based single speaker speech synthesis outperforms deep neural network (DNN)-based ones in the framework of statistical parametric speech synthesis. By extending this method to multiple speakers, it is expected that higher speech quality can be achieved with a smaller number of training utterances from each speaker. To apply DGPs to multi-speaker speech synthesis, we propose two methods: one using DGP with one-hot speaker codes, and the other using a deep Gaussian process latent variable model (DGPLVM). The DGP with one-hot speaker codes uses additional GP layers to transform speaker codes into latent speaker representations. The DGPLVM directly models the distribution of latent speaker representations and learns it jointly with acoustic model parameters. In this method, acoustic speaker similarity is expressed in terms of the similarity of the speaker representations, and thus, the voices of similar speakers are efficiently modeled. We experimentally evaluated the performance of the proposed methods in comparison with those of conventional DNN and variational autoencoder (VAE)-based frameworks, in terms of acoustic feature distortion and subjective speech quality. The experimental results demonstrate that (1) the proposed DGP-based and DGPLVM-based methods improve subjective speech quality compared with a feed-forward DNN-based method, and (2) even when the amount of training data for target speakers is limited, the DGPLVM-based method outperforms other methods, including the VAE-based one. Additionally, (3) by using a speaker representation randomly sampled from the learned speaker space, the DGPLVM-based method can generate voices of non-existent speakers.

Speech signal enhancement in cocktail party scenarios by deep learning based virtual sensing of head-mounted microphones

The cocktail party effect refers to the human sense of hearing’s ability to pay attention to a single conversation while filtering out all other background noise. To mimic this human hearing ability for people with hearing loss, scientists integrate beamforming algorithms into the signal processing path of hearing aids or implants’ audio processors. Although these algorithms’ performance strongly depends on the number and spatial arrangement of the microphones, most devices are equipped with a small number of microphones mounted close to each other on the audio processor housing. We measured and evaluated the impact of the number and spatial arrangement of hearing aid or head-mounted microphones on the performance of the established Minimum Variance Distortionless Response beamformer in cocktail party scenarios. The measurements revealed that the optimal microphone placement exploits monaural cues (pinna-effect), is close to the target signal, and creates a large distance spread due to its spatial arrangement. However, this microphone placement is impractical for hearing aid or implant users, as it includes microphone positions such as on the forehead. To overcome microphones’ placement at impractical positions, we propose a deep virtual sensing estimation of the corresponding audio signals. The results of objective measures and a subjective listening test with 20 participants showed that the virtually sensed microphone signals significantly improved the speech quality, especially in cocktail party scenarios with low signal-to-noise ratios. Subjective speech quality was assessed using a 3-alternative forced choice procedure to determine which of the presented speech mixtures was most pleasant to understand. Hearing aid and cochlear implant (CI) users might benefit from the presented approach using virtually sensed microphone signals, especially in noisy environments.

Speech quality estimation with deep lattice networks.

Intrusive subjective speech quality estimation of mean opinion score (MOS) often involves mapping a raw similarity score extracted from differences between the clean and degraded utterance onto MOS with a fitted mapping function. More recent models such as support vector regression (SVR) or deep neural networks use multidimensional input, which allows for a more accurate prediction than one-dimensional (1-D) mappings but does not provide the monotonic property that is expected between similarity and quality. We investigate a multidimensional mapping function using deep lattice networks (DLNs) to provide monotonic constraints with input features provided by ViSQOL. The DLN improved the speech mapping to 0.24 mean-square error on a mixture of datasets that include voice over IP and codec degradations, outperforming the 1-D fitted functions and SVR as well as PESQ and POLQA. Additionally, we show that the DLN can be used to learn a quantile function that is well-calibrated and a useful measure of uncertainty. The quantile function provides an improved mapping of data driven similarity representations to human interpretable scales, such as quantile intervals for predictions instead of point estimates.

Use of Emotional and Neutral Speech in Evaluating Compression Speeds.

Emotional speech differs from neutral speech in its envelope characteristics. Use of emotional speech materials may be more sensitive for evaluating signal processing algorithms that affect the temporal envelope. Subjective listener preference was compared between variable speed compression (VSC) and fast acting compression (FAC) amplitude compression algorithms using neutral and emotional speech. The study used a single-blinded, repeated measures design. Twenty hearing-impaired (HI) listeners with a bilaterally symmetrical, mild- to-moderately severe sensorineural hearing loss and 21 listeners with normal hearing (NH) participated. Speech was processed using FAC and VSC algorithms. A paired-comparison paradigm assessed subjective preference for FAC versus VSC using emotional and neutral speech materials. The significance of subjective preference for compression algorithm (FAC or VSC) was evaluated using a linear mixed effects model at each combination of stimulus type (emotional or neutral speech) and hearing group (NH or HI). HI listeners showed a preference for VSC over FAC when listening to emotional speech. The same listeners showed a nonsignificant, preference for VSC over FAC when listening to neutral speech. NH listeners showed preference for VSC over FAC for both neutral and emotional speech materials. These results suggest that the subjective sound quality of emotional speech is more susceptible than neutral speech to changes in the signal introduced by FAC. Clinicians should consider including emotional speech materials when evaluating listener preference for different compression speeds in the clinic.

Noise and acoustic conditions of premises for hearing-impaired people in Korea

Hearing-impaired people need more stringent acoustic and noise requirements than normal-hearing people in terms of speech intelligibility and listening effort. Multiple guidelines recommend a maximum reverberation time of 0.4 s in classrooms, signal-to-noise ratios (SNRs) greater than 15 dB, and ambient noise levels lower than 35 dBA. We measured noise levels and room acoustic parameters of 12 classrooms in two schools for hearing-impaired pupils, a dormitory apartment for the hearing-impaired, and a church mainly for the hearing-impaired in the Republic of Korea. Additionally, subjective speech clarity and quality of verbal communication were evaluated through questionnaires and interviews with hearing-impaired students in one school. Large differences in subjective speech perception were found between younger primary school pupils and older pupils. Subjective data from the questionnaire and interview were inconsistent; major challenges in obtaining reliable subjective speech perception and limitations of the results are discussed.

Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods

Subjective speech quality assessment has traditionally been carried out in laboratory environments under controlled conditions. With the advent of crowdsourcing platforms tasks, which need human intelligence, can be resolved by crowd workers over the Internet. Crowdsourcing also offers a new paradigm for speech quality assessment, promising higher ecological validity of the quality judgments at the expense of potentially lower reliability. This paper compares laboratory-based and crowdsourcing-based speech quality assessments in terms of comparability of results and efficiency. For this purpose, three pairs of listening-only tests have been carried out using three different crowdsourcing platforms and following the ITU-T Recommendation P.808. In each test, listeners judge the overall quality of the speech sample following the Absolute Category Rating procedure. We compare the results of the crowdsourcing approach with the results of standard laboratory tests performed according to the ITU-T Recommendation P.800. Results show that in most cases, both paradigms lead to comparable results. Notable differences are discussed with respect to their sources, and conclusions are drawn that establish practical guidelines for crowdsourcing-based speech quality assessment.

Sound Quality Effects of an Adaptive Nonlinear Frequency Compression Processor with Normal-Hearing and Hearing-Impaired Listeners.

Frequency lowering (FL) technology offers a means of improving audibility of high-frequency sounds. For some listeners, the benefit of such technology can be accompanied by a perceived degradation in sound quality, depending on the strength of the FL setting. The studies presented in this article investigate the effect of a new type of FL signal processing for hearing aids, adaptive nonlinear frequency compression (ANFC), on subjective speech quality. Listener ratings of sound quality were collected for speech stimuli processed with systematically varied fitting parameters. Study 1 included 40 normal-hearing (NH) adult and child listeners. Study 2 included 11 hearing-impaired (HI) adult and child listeners. HI listeners were fitted with laboratory-worn hearing aids for use during listening tasks. Speech quality ratings were assessed across test conditions consisting of various strengths of static nonlinear frequency compression (NFC) and ANFC speech. Test conditions included those that were fine-tuned on an individual basis per hearing aid fitting and conditions that were modified to intentionally alter the sound quality of the signal. Listeners rated speech quality using the MUlti Stimulus test with Hidden Reference and Anchor (MUSHRA) test paradigm. Ratings were analyzed for reliability and to compare results across conditions. Results show that interrater reliability is high for both studies, indicating that NH and HI listeners from both adult and child age groups can reliably complete the MUSHRA task. Results comparing sound quality ratings across experimental conditions suggest that both the NH and HI listener groups rate the stimuli intended to have poor sound quality (e.g., anchors and the strongest available parameter settings) as having below-average sound quality ratings. A different trend in the results is reported when considering the other experimental conditions across the listener groups in the studies. Speech quality ratings measured with NH listeners improve as the strength of ANFC decreases, with a range of bad to good ratings reported, on average. Speech quality ratings measured with HI listeners are similar and above-average for many of the experimental stimuli, including those with fine-tuned NFC and ANFC parameters. Overall, HI listeners provide similar sound quality ratings when comparing static and adaptive forms of frequency compression, especially when considering the individualized parameter settings. These findings suggest that a range in settings may result in above-average sound quality for adults and children with hearing impairment. Furthermore, the fitter should fine-tune FL parameters for each individual listener, regardless of type of FL technology.

Objective and Subjective Speech Quality Assessment of Amplification Devices for Patients With Parkinson's Disease.

This paper investigated subjective and objective assessment of Parkinsonian speech quality. Speech stimuli were recorded from 11 Parkinsonian and 10 age-matched normal control participants under different amplification and environmental conditions. Quality ratings of the recorded stimuli were obtained from naïve listeners. For objective assessment, feature vectors were derived from the speech recordings based on temporal, spectral, and/or cepstral parametrization. These feature vectors were subsequently mapped to the predicted quality scores through several regression methods, including support vector regression, Gaussian process regression, and deep learning. Analyses of subjective speech quality ratings showed that Parkinsonian speech quality was significantly poorer than control subjects' speech quality, and that the amplification devices differentially affected perceived quality of Parkinsonian speech. Objective analyses revealed disparity in performance among feature vectors and mappers, with some feature vector and mapper combinations exhibiting statistically similar correlations with subjective ratings. A set consisting of cepstral, spectral, and modulation domain speech features when combined with Gaussian process regression or deep learning resulted in the highest correlation of 0.85 with the subjective data.

Subjective Self-Rated Speech Intelligibility and Quality of Life in Patients with Parkinson’s Disease in a Malaysian Sample

Objectives: This study compared the Quality of Life (QOL) of individuals with Parkinson’s disease (PD) with a healthy control from a sample in Malaysia. The relationship between the QOL, age, gender, duration of disease, self-rated speech scale, and life satisfaction level was examined. Methods: Fifty-four individuals with PD and 55 non-PD control adults completed the Parkinson’s Disease Questionnaire-39 (PDQ-39) and rated their speech intelligibility [Self-Rated Speech Scale (SRSS)] and life satisfaction level [Life Satisfaction Scale (LSS)] on a 10-point scale. Results: PD participants reported significantly lower QOL scores than the control group (Mann-Whitney U). Although there was no significant relationship between the QOL, age, gender, and duration of disease for PD participants, moderately significant correlations were observed between QOL, LSS and SRSS ratings (Spearman correlation). Individuals with PD showed lower QOL and higher concern in their communication than the control group. Conclusion: Identifying patients’ perception of their speech performance could help clinicians to better understand patients’ needs when delivering speech therapy services.

Subjective speech quality measurement with and without parallel task: Laboratory test results comparison

This paper focuses on a novel methodology of subjective speech quality measurement and repeatability of its results between laboratory conditions and simulated environmental conditions. A single set of speech samples was distorted by various background noises and low bit-rate coding techniques. This study aimed to compare results of subjective speech quality tests with and without a parallel task deploying the ITU-T P.835 methodology. Afterward, tests results performed with and without a parallel task were compared using Pearson correlation, CI95, and numbers of opposite pair-wise comparisons. The tests show differences in results in the case of a parallel task.

Investigating different representations for modeling and controlling multiple emotions in DNN-based speech synthesis

In this paper, we investigate the simultaneous modeling of multiple emotions in DNN-based expressive speech synthesis, and how to represent the emotional labels, such as emotional class and strength, for this task. Our goal is to answer two questions: First, what is the best way to annotate speech data with multiple emotions – should we use the labels that the speaker intended to express, or labels based on listener perception of the resulting speech signals? Second, how should the emotional information be represented as labels for supervised DNN training, e.g., should emotional class and emotional strength be factorized into separate inputs or not? We evaluate on a large-scale corpus of emotional speech from a professional voice actress, additionally annotated with perceived emotional labels from crowdsourced listeners. By comparing DNN-based speech synthesizers that utilize different emotional representations, we assess the impact of these representations and design decisions on human emotion recognition rates, perceived emotional strength, and subjective speech quality. Simultaneously, we also study which representations are most appropriate for controlling the emotional strength of synthetic speech.

Spectral difference for statistical model-based speech enhancement in speech recognition

In this paper, we propose a statistical model-based speech enhancement technique using the spectral difference scheme for the speech recognition in virtual reality. In the analyzing step, two principal parameters, the weighting parameter in the decision-directed (DD) method and the long-term smoothing parameter in noise estimation, are uniquely determined as optimal operating points according to the spectral difference under various noise conditions. These optimal operating points, which are specific according to different spectral differences, are estimated based on the composite measure, which is a relevant criterion in terms of speech quality. An efficient mapping function is also presented to provide an index of the metric table associated with the spectral difference so that operating points can be determined according to various noise conditions for an on-line step. In the on-line speech enhancement step, different parameters are chosen on a frame-by-frame basis under the metric table of the spectral difference. The performance of the proposed method is evaluated using objective and subjective speech quality measures in various noise environments. Our experimental results show that the proposed algorithm yields better performances than conventional algorithms.

Subjective speech quality measurement repeatability: comparison of laboratory test results

This article reports on a multi-lab subjective listening experiment aiming at inter-lab and intra-lab test results repeatability verification. An identical set of speech samples corresponding to contemporary networks has been tested by three independent labs deploying ITU-T P.835 methodology. The tests results have been compared regarding Pearson correlation, RMSE, RMSE* and numbers of opposite pair-wise comparisons. The results show the level of inter-lab and intra-lab repeatability in the case of identical test speech samples utilization and thus confirm the subjective tests are highly repeatable in case they follow recommendation requirements strictly. The tests also show differences in results in case subject expectations are set differently using a wider set of test speech samples (as presented in one of the labs).

Novel adaptive muting technique for packet loss concealment of ITU-T G.722 using optimized parametric shaping functions

Adaptive muting method using an optimized parametric shaping function as a part of the ITU-T G.722 Appendix IV packet loss concealment algorithm is proposed. The packet loss concealment algorithm incorporating an adaptive muting scheme is known to prevent the generation of unpleasant sounds during packet loss concealment. However, original muting uses a piece-wise linear muting curve according to packet errors so that our muting approach uses non-linear parametric shaping functions including sigmoid and raised-cosine. Training is substantially performed to determine the parameters of the two shaping functions in terms of objective speech quality measures, and optimal parameters are finally selected by subjective speech quality. Through extensive experiments, this proposed muting technique turns out to improve the performance of the reference muting mechanisms in the packet loss concealment algorithm of the G.722 Appendix IV under various experimental conditions.

On using multivariate polynomial regression model with spectral difference for statistical model-based speech enhancement

In this paper, we propose a statistical model-based speech enhancement technique using a multivariate polynomial regression (MPR) based on spectral difference scheme. In the analyzing step, three principal parameters, the weighting parameter in the decision-directed (DD) method, the long-term smoothing parameter for the noise estimation, and the control parameter of the minimum gain value are estimated as optimal operating points technique by using to the spectral difference under various noise conditions. These optimal operating points, which are specific according to different spectral differences, are estimated based on the composite measure, which is a relevant criterion in terms of speech quality. Thus, we apply the MPR technique by incorporating the spectral differences as independent variables in order to estimate the optimal operating points. The MPR technique offers an effective scheme to represent complex nonlinear input-output relationship between the optimal operating points and spectral differences so that operating points can be determined according to various noise conditions in the off-line step. In the on-line speech enhancement step, different parameters are chosen on a frame-by-frame basis through the regression according to the spectral difference. The performance of the proposed method is evaluated using objective and subjective speech quality measures in various noise environments. Our experimental results show that the proposed algorithm yields better performances than conventional algorithms.

Evaluation of VoIP Speech Quality Using Neural Network

The automatic quantification of the speech quality is an important and unsolved problem. The ability this task to be executed in real time is particularly interesting. Many parameters affect the voice quality, but only subjective assessments are decisive. The article makes a proposal to create cost effective and efficient solution for prediction the subjective speech quality assessment. A neural network is used as a customer. For training of the neural network, BYE Statistics is used. It gives the client's view for the voice quality. Result of some practical experiments is given in article.

The relationship between fluency, intelligibility, and acceptability of non-native spoken English

Non-native accented speech is typically less intelligible and less fluent than native speech, but it is unclear how these factors interact to influence perceived speech quality. To investigate this question, the speech of 20 non-native speakers of English varying in proficiency and native language was evaluated. Subjective measures of speech quality (listening effort, acceptability, and intelligibility) were compared to objective measures of word recognition by native listeners, and to acoustic measures of fluency and of segmental and suprasegmental properties related to intelligibility. Results showed that subjective quality measures were highly related to one another and to word recognition and were most strongly predicted by measures of fluency. Segmental and suprasegmental measures did not predict word recognition or subjective speech quality. There was also an interaction between the effects of proficiency and speaker's native language on word recognition, but this did not extend to subjective measures. Finally, listeners who first heard high-proficiency speakers gave overall lower subjective quality ratings but there was no interaction between proficiency and presentation order. Multivariate analyses suggest that factors related to speaking rate, including pause duration, have the greatest effect on measures of acceptability, intelligibility, and listening effort. [Work supported by Purdue Linguistics and Purdue Research Foundation.]

Comparison of two channel selection criteria for noise suppression in cochlear implants

The performance of current channel selection criteria used in cochlear implant (CI) devices (e.g., maximum selection criterion used in ACE) degrades significantly in the presence of noise. In noisy backgrounds, coding strategies that select the "right" channels for stimulation could potentially produce substantial improvements in intelligibility. In this study, the performance of two alternative channel selection criteria is assessed in terms of intelligibility and subjective quality with CI users in noise. The performance is compared against that of the maximum selection scheme employed in the ACE strategy (comparison is also made with the CIS strategy). Sentences were presented to seven CI users in speech weighted noise (-5, 0, and 5 dB SNR). Both channel selection criteria were implemented under ideal conditions where a priori knowledge of the target and masker was assumed. This was done to assess the full potential benefit of selecting the "right" channels for stimulation in noisy backgrounds. Substantial intelligibility improvement relative to the CI users' daily strategy (i.e., ACE or CIS) was achieved with the two different channel selection criteria under all noisy conditions considered. No significant difference in subjective quality of noisy speech processed by the two channel selection criteria was observed.

Compressed domain speech enhancement method based on ITU-T G.722.2

Based on the bit-stream of ITU-T G.722.2 speech coding standard, through the modification of codebook gains in the codec, a compressed domain speech enhancement method that is compatible with the discontinuous transmission (DTX) mode and frame erasure condition is proposed in this paper. In non-DTX mode, the Voice Activity Detection (VAD) is carried out in the compressed domain, and the background noise is classified into full-band distributed noise and low-frequency distributed noise. Then, the noise intensity is estimated based on the algebraic codebook power, and the a priori SNR is estimated according to the noise type. Next, the codebook gains are jointly modified under the rule of energy compensation. Especially, the adaptive comb filter is adopted to remove the residual noise in the excitation signal in low-frequency distributed noise. Finally, the modified codebook gains are re-quantized in speech or excitation domain. For non-speech frames in DTX mode, the logarithmic frame energy is attenuated to remove the noise, while the spectral envelope is kept unchanged. When frame erasure occurs, the recovered algebraic codebook gain is exponentially attenuated, and based on the reconstructed algebraic codebook vector, all the codec parameters are re-quantized to form the error concealed bit-stream. The result of performance evaluation under ITU-T G.160 shows that, with much lower computational complexity, better noise reduction, SNR improvement, and objective speech quality performances are achieved by the proposed method comparing with the state-of-art compressed domain methods. The subjective speech quality test shows that, the speech quality of the proposed method is better than the method that only modifies the algebraic codebook gain, and similar to the one with the assistance of linear domain speech enhancement method.