Speech Quality Assessment Research Articles

Objective approach in the assessment of speech quality in patients with tongue cancer after hemiglossectomy

Introduction. Surgery is the leading treatment for tongue cancer, performance which leads to impaired speech function. Because speech is one of the most important tools of social interaction, its disorder can significantly affect patients’ quality of life. To date, there is no generally accepted objective approach to assessing the quality of speech that would make it possible to analyze speech changes at different stages of therapy and help in development of treatment strategy.Aim. To compare the sound reproduction of patients with tongue cancer after hemiglossectomy with and without reconstruction using an objective assessment method, namely – a special software.Materials and methods. The study included 29 patients who received surgical treatment in the volume of hemiglossectomy with ipsilateral cervical lymphodissection. The patients were divided into 2 groups: the group 1 included 14 patients who underwent reconstruction with submental, radial and buccal flaps, the group 2 – 15 patients who did not underwent it. Sound reproduction by each patient was recorded before the operation, no earlier than 10–12 days after it (session of type 1) and after the end of specialized treatment: radiation therapy, chemoradiation therapy (session of type 2). If adjuvant therapy was not prescribed, the type 2 session was performed 3–6 months after surgery. The records made during sessions of the types 1 and 2 were compared with records of preoperative session. During each session, the pronunciation of 30 syllables containing 3 problematic sounds was recorded: [k], [s] and [t] (10 syllables with each of the sounds). The differences in the data obtained during the sessions types 1 and 2 were analyzed according both the syllable set and separately for each sound studied.Results. The reconstruction performed improves sound pronunciation [t] after hemiglossectomy both after surgery and 3–6 months after the end of specialized treatment (p <0.01). After completion of specialized treatment, the pronunciation of the sound [s] significantly improved in both groups. However, in the reconstruction group, the variations in pronunciation values of this sound before the operation and after it obtained in the groups 1 and 2 were significantly reduced. During the assessment of the pronunciation of sound [k], the similar data were obtained. When comparing the quality of speech of patients during the type 2 session, the best results were obtained in group 1: in patients of this group, there were smaller mean differences in the sounds pronunciation as compared to the initial recording (p <0.001) than in patients of group 2.Conclusion. Resection of the tongue most negatively affects pronunciation of the dental-alveolar sounds than the whistling or palatine-dental sounds. The time factor has a positive effect on the speech quality of patients, therefore, the assessment of sound reproduction in patients after hemiglossectomy should be performed no earlier than 3–6 months after the end of specialized treatment.

ARCnet: A Multi-Feature-Based Auto Radio Check Model

Radio checks serve as the foundation for ground-to-air communication. To integrate machine learning for automated and reliable radio checks, this study introduces an Auto Radio Check network (ARCnet), a novel algorithm for non-intrusive speech quality assessment in civil aviation, addressing the crucial need for dependable ground-to-air communication. By employing a multi-scale feature fusion approach, including the consideration of audio’s frequency domain, comprehensibility, and temporal information within the radio check scoring network, ARCnet integrates manually designed features with self-supervised features and utilizes a transformer network to enhance speech segment analysis. Utilizing the NISQA open-source dataset and the proprietary RadioCheckSpeech dataset, ARCnet demonstrates superior performance in predicting speech quality, showing a 12% improvement in both the Pearson correlation coefficient and root mean square error (RMSE) compared to existing models. This research not only highlights the significance of applying multi-scale attributes and deep neural network parameters in speech quality assessment but also emphasizes the crucial role of the temporal network in capturing the nuances of voice data. Through a comprehensive comparison of the ARCnet approach to traditional methods, this study underscores its innovative contribution to enhancing communication efficiency and safety in civil aviation.

Research on Oral English Learning System Integrating AI Speech Data Recognition and Speech Quality Evaluation Algorithm

The application of AI speech data recognition (SDR) and speech quality assessment algorithms in spoken English learning systems can help to realize the intelligence and automation of the process related to spoken English learning. To better assist the people concerned with the learning of spoken English, it is necessary to integrate AI speech data recognition technology in the process of learning spoken English to bring learners a learning environment and learning conditions that are not bound by time and space. As an important way to realize the natural communication of information between machines and people, AI speech recognition technology has significant application advantages and application values in eliminating the influence of the native language of English speaking learners and innovating the learning mode in the local area. In addition, the assessment of the reliability of spoken English language learning can be carried out in several dimensions. However, it is still difficult to unify the criteria of many sizes, bringing new application scenarios for AI SDR. Based on this, this paper first analyzes the theoretical basis of AI SDR. Then it investigates the speech data recognition method of the fused AI English-speaking learning mechanism and the process of fused AI English-speaking speech quality assessment. Finally, it designs and verifies the functions of the spoken English learning mechanism incorporating AI speech data identification and quality assessment.

Non-intrusive speech quality assessment: A survey

Speech quality is a critical consideration for applications such as speech enhancement, coding, transmission, and synthesis. Accurately evaluating the quality of degraded speech without a reference is particularly challenging. As a result, non-intrusive speech quality assessment has been extensively researched due to its efficiency and ease of deployment. This survey comprehensively reviews these studies. Commonly used speech quality assessment databases are reviewed, as they serve as validation sets for objective metrics. Non-intrusive speech quality assessment methods are categorized and reviewed, and the performance of state-of-the-art and classical speech quality assessment methods is compared. The survey presents an overview of classic algorithms and recent advancements in the field of non-intrusive speech quality assessment.

Non-Intrusive Speech Quality Assessment Based on Deep Neural Networks for Speech Communication.

Traditionally, speech quality evaluation relies on subjective assessments or intrusive methods that require reference signals or additional equipment. However, over recent years, non-intrusive speech quality assessment has emerged as a promising alternative, capturing much attention from researchers and industry professionals. This article presents a deep learning-based method that exploits large-scale intrusive simulated data to improve the accuracy and generalization of non-intrusive methods. The major contributions of this article are as follows. First, it presents a data simulation method, which generates degraded speech signals and labels their speech quality with the perceptual objective listening quality assessment (POLQA). The generated data is proven to be useful for pretraining the deep learning models. Second, it proposes to apply an adversarial speaker classifier to reduce the impact of speaker-dependent information on speech quality evaluation. Third, an autoencoder-based deep learning scheme is proposed following the principle of representation learning and adversarial training (AT) methods, which is able to transfer the knowledge learned from a large amount of simulated speech data labeled by POLQA. With the help of discriminative representations extracted from the autoencoder, the prediction model can be trained well on a relatively small amount of speech data labeled through subjective listening tests. Fourth, an end-to-end speech quality evaluation neural network is developed, which takes magnitude and phase spectral features as its inputs. This phase-aware model is more accurate than the model using only the magnitude spectral features. A large number of experiments are carried out with three datasets: one simulated with labels obtained using POLQA and two recorded with labels obtained using subjective listening tests. The results show that the presented phase-aware method improves the performance of the baseline model and the proposed model with latent representations extracted from the adversarial autoencoder (AAE) outperforms the state-of-the-art objective quality assessment methods, reducing the root mean square error (RMSE) by 10.5% and 12.2% on the Beijing Institute of Technology (BIT) dataset and Tencent Corpus, respectively. The code and supplementary materials are available at https://github.com/liushenme/AAE-SQA.

EMDSQA: A Neural Speech Quality Assessment Model with Speaker Embedding

EMDSQA: A Neural Speech Quality Assessment Model with Speaker Embedding

Non-intrusive assessment of speech quality for classical Chinese poetry recitals using perceptual and acoustic features

Non-intrusive speech quality assessment is vital for objectively rating vast speech datasets without pristine reference signals. However, most existing research emphasizes speech clarity, neglecting tempo and rhythm. Different from normal Chinese speech, the prosody of classical Chinese poetry conveys profound emotion and a unique classical beauty. To assess the speech quality of classical Chinese poetry recitals across varied reciters and recording conditions, it's essential to design modules evaluating both clarity and prosody. In this paper, we propose a non-intrusive assessment method, utilizing perceptual and acoustic features as key indicators. First, by extracting the poetry's pitch frequency, we quantify the reference assessment function, deriving a prosodic score based on deviation degrees. Secondly, we devise a residual-structured neural network to capture the Mel-spectrogram's perceptual features, yielding the objective MOS score. Finally, we craft a comprehensive model using second-order polynomial regression to map both the objective MOS and prosody scores to the poetry recital quality score. After experimental validation, prosody and clarity modules both show outstanding prediction accuracy. The final quality assessment score performs a high correlation with human perception.

New research on monaural speech segregation based on quality assessment

Speech enhancement (SE) is a pivotal technology in enhancing the quality and intelligibility of speech signals. Nevertheless, when processing speech signals under conditions of high signal-to-noise ratio (SNR), conventional SE techniques may inadvertently lead to a diminution in the perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). This article introduces the innovative incorporation of the Non-Intrusive Speech Quality Assessment (NISQA) algorithm into SE systems. Through the comparison of pre and post-enhancement speech quality scores, it discerns whether the speech signal under consideration warrants enhancement processing, thereby mitigating potential deterioration in PESQ and STOI. Furthermore, this study delves into the ramifications of five prevalent speech features, namely, Mel Frequency Cepstral Coefficients (MFCC), Gammatone Frequency Cepstral Coefficients (GFCC), Relative Spectral Trans-formed Perceptual Linear Prediction coefficients (RASTA-PLP), Amplitude Modulation Spectrogram (AMS), and Multi-Resolution Cochleagram (MRCG), on PESQ and STOI under varying noise conditions. Experimental outcomes underscore that MRCG consistently emerges as the optimal and most stable feature for STOI, while the feature yielding the highest PESQ score exhibits intricate correlations with the background noise type, SNR level, and noise compatibility with the speech signal. Consequently, we propose an SE methodology founded on quality assessment and feature selection, facilitating the adaptive selection of optimal features tailored to distinct background noise scenarios, thereby always maintain the highest caliber enhancement effect with regard to PESQ metrics.

Non-Intrusive Air Traffic Control Speech Quality Assessment with ResNet-BiLSTM

In the current field of air traffic control speech, there is a lack of effective objective speech quality evaluation methods. This paper proposes a new network framework based on ResNet–BiLSTM to address this issue. Firstly, the mel-spectrogram of the speech signal is segmented using the sliding window technique. Next, a preceding feature extractor composed of convolutional and pooling layers is employed to extract shallow features from the mel-spectrogram segment. Then, ResNet is utilized to extract spatial features from the shallow features, while BiLSTM is used to extract temporal features, and these features are horizontally concatenated. Finally, based on the concatenated spatiotemporal features, the final speech quality score is computed using fully connected layers. We conduct experiments on the air traffic control speech database and compare the objective scoring results with the subjective scoring results. The experimental results demonstrate that the proposed method has a high correlation with the mean opinion score (MOS) of air traffic control speech.

MSQAT: A multi-dimension non-intrusive speech quality assessment transformer utilizing self-supervised representations

Convolutional neural networks (CNNs) have been widely utilized as the main building block for many non-intrusive speech quality assessment (NISQA) methods. A new trend is to add a self-attention mechanism based on CNN to better capture long-term global content. However, it is not clear whether the pure attention-based network is sufficient to obtain good performance in NISQA. To this end, a framework named Multi-dimension non-intrusiveSpeech Quality Assessment Transformer (MSQAT) is proposed. To strengthen the interactions of various speech regions between local and global, we proposed the Audio Spectrogram Transformer Block (ASTB), Transposed Attention Block (TAB) and the Residual Swin Transformer Block (RSTB). These three modules employ attention mechanisms across spatial and channel dimensions, respectively. Additionally, speech quality varies not only in different frames, but also in different frequencies. Thus, a two-branch structure is designed to better evaluate the quality of speech by considering the weighting of each patch's score. Experimental results demonstrate that the proposed MSQAT has state-of-the-art performance on three standard datasets (NISQA Corpus, Tencent Corpus, and PSTN Corpus) and indicate that the pure attention model can achieve or surpass the performance of other CNN-attention hybrid models.

Implementation and evaluation of low-speed speech compression LPC and MELP algorithms

Abstract— Currently, the design of a speech signal compression module that responds to low speed but still ensures signal quality for communication devices is a topic of interest for research. LPC (Linear Prediction Coding) and MELP (Mixed-Excitation Linear Prediction), two common approaches, are extensively researched and used, particularly in the vocoder of radio communication equipment like HF or VHF equipment. Implementing security solutions for these devices are challenging due to the crucial role played by the vocoder in signal processing prior to encryption. The paper first analyzes the two algorithms LPC and MELP and implements them on the current popular hardware technologies ARM and DSP to give an evaluation direction when choosing technology to serve the design of vocoders. There are many different methods of speech quality assessment in this paper, the PESQ method according to the ITU-T P862 standard was used to evaluate the results of the implementation of the proposed algorithms. The results show that the quality of the compression rate reduction of about 2 kbps is still satisfactory, and dedicated DSP technology for better speech quality results is a basis for helping vocoder device developers choose the right hardware technology when designing products.

NISQE: Non-Intrusive Speech Quality Evaluator Based on Natural Statistics of Mean Subtracted Contrast Normalized Coefficients of Spectrogram

With the evolution in technology, communication based on the voice has gained importance in applications such as online conferencing, online meetings, voice-over internet protocol (VoIP), etc. Limiting factors such as environmental noise, encoding and decoding of the speech signal, and limitations of technology may degrade the quality of the speech signal. Therefore, there is a requirement for continuous quality assessment of the speech signal. Speech quality assessment (SQA) enables the system to automatically tune network parameters to improve speech quality. Furthermore, there are many speech transmitters and receivers that are used for voice processing including mobile devices and high-performance computers that can benefit from SQA. SQA plays a significant role in the evaluation of speech-processing systems. Non-intrusive speech quality assessment (NI-SQA) is a challenging task due to the unavailability of pristine speech signals in real-world scenarios. The success of NI-SQA techniques highly relies on the features used to assess speech quality. Various NI-SQA methods are available that extract features from speech signals in different domains, but they do not take into account the natural structure of the speech signals for assessment of speech quality. This work proposes a method for NI-SQA based on the natural structure of the speech signals that are approximated using the natural spectrogram statistical (NSS) properties derived from the speech signal spectrogram. The pristine version of the speech signal follows a structured natural pattern that is disrupted when distortion is introduced in the speech signal. The deviation of NSS properties between the pristine and distorted speech signals is utilized to predict speech quality. The proposed methodology shows better performance in comparison to state-of-the-art NI-SQA methods on the Centre for Speech Technology Voice Cloning Toolkit corpus (VCTK-Corpus) with a Spearman’s rank-ordered correlation constant (SRC) of 0.902, Pearson correlation constant (PCC) of 0.960, and root mean squared error (RMSE) of 0.206. Conversely, on the NOIZEUS-960 database, the proposed methodology shows an SRC of 0.958, PCC of 0.960, and RMSE of 0.114.

Non-intrusive speech quality assessment with attention-based ResNet-BiLSTM.

Speech quality is frequently affected by a variety factors in online conferencing applications, such as background noise, reverberation, packet loss and network jitter. In real scenarios, it is impossible to obtain a clean reference signal for evaluating the quality of the conferencing speech. Therefore, an effective non-intrusive speech quality assessment (NISQA) method is necessary. In this paper, we propose a new network framework for NISQA based on ResNet and BiLSTM. ResNet is utilized to extract local features, while BiLSTM is used to integrate representative features with long-term time dependencies and sequential characteristics. Considering that ResNet may result in the loss of context information when applied to the NISQA task, we propose a variant of ResNet which can preserve the time series information of the conferencing speech. The experimental results demonstrate that the proposed method has a high correlation with the mean opinion score of clean, noisy and processed speech.

Differentiable Mean Opinion Score Regularization for Perceptual Speech Enhancement

Many speech enhancement methods require perceptual quality metrics for evaluation. The “holy grail” of perceptual speech quality assessment is human subjective ratings, known as the mean opinion score. However, acquiring human ratings is time-consuming, laborious, and expensive. Existing objective quality metrics, on the other hand, are efficient and easy to compute but do not correlate well with human ratings. In this paper, we propose a relatively lightweight deep-learning-based model to predict the human ratings of speech signals. Since it is differentiable, it can be easily employed as a perceptual regularization to improve existing deep-learning-based speech enhancement methods. Experimental results demonstrate that the predictions of our proposed model correlate well with human judgments. We present application in speech enhancement and show that, interestingly, while there is a degradation in performance in terms of traditional objective metrics, there is a significant improvement in the perceptual quality and the naturalness of the enhanced speech.

Deep Learning-Based Non-Intrusive Multi-Objective Speech Assessment Model With Cross-Domain Features

This study proposes a cross-domain multi-objective speech assessment model, called MOSA-Net, which can simultaneously estimate the speech quality, intelligibility, and distortion assessment scores of an input speech signal. MOSA-Net comprises a convolutional neural network and bidirectional long short-term memory architecture for representation extraction, and a multiplicative attention layer and a fully connected layer for each assessment metric prediction. Additionally, cross-domain features (spectral and time-domain features) and latent representations from self-supervised learned (SSL) models are used as inputs to combine rich acoustic information to obtain more accurate assessments. Experimental results show that in both seen and unseen noise environments, MOSA-Net can improve the linear correlation coefficient (LCC) scores in perceptual evaluation of speech quality (PESQ) prediction, compared to Quality-Net, an existing single-task model for PESQ prediction, and improve LCC scores in short-time objective intelligibility (STOI) prediction, compared to STOI-Net, an existing single-task model for STOI prediction. Moreover, MOSA-Net can be used as a pre-trained model to be effectively adapted to an assessment model for predicting subjective quality and intelligibility scores with a limited amount of training data. Experimental results show that MOSA-Net can improve LCC scores in mean opinion score (MOS) predictions, compared to MOS-SSL, a strong single-task model for MOS prediction. We further adopt the latent representations of MOSA-Net to guide the speech enhancement (SE) process and derive a quality-intelligibility (QI)-aware SE (QIA-SE) approach. Experimental results show that QIA-SE outperforms the baseline SE system with improved PESQ scores in both seen and unseen noise environments over a baseline SE model.

Method to control the amount of “musical” noise for speech quality assessments

This study presents a method of adding to clean speech signals a controlled degree of “musical” noise distortions that mimic typical artefacts of speech enhancement systems. The resulting distorted speech signals were evaluated with respect to listening effort and sound quality in subjective listening tests and via model predictions. Both subjective ratings and model prediction outcomes covered the entire rating scale from “excellent”/ “no effort” to “bad”/ “extreme effort”, respectively, in a consistent way. The proposed method proved to be useful for systematic assessments of “musical” noise distortions for the conditions tested in this study.

Non-intrusive speech quality assessment using context-aware neural networks

To meet the human perceived quality of experience (QoE) while communicating over various Voice over Internet protocol (VoIP) applications, for example Google Meet, Microsoft Skype, Apple FaceTime, etc. a precise speech quality assessment metric is needed. The metric should be able to detect and segregate different types of noise degradations present in the surroundings before measuring and monitoring the quality of speech in real-time. Our research is motivated by the lack of clear evidence presenting speech quality metric that can firstly distinguish different types of noise degradations before providing speech quality prediction decision. To that end, this paper presents a novel non-intrusive speech quality assessment metric using context-aware neural networks in which the noise class (context) of the degraded or noisy speech signal is first identified using a classifier then deep neutral networks (DNNs) based speech quality metrics (SQMs) are trained and optimized for each noise class to obtain the noise class-specific (context-specific) optimized speech quality predictions (MOS scores). The noisy speech signals, that is, clean speech signals degraded by different types of background noises are taken from the NOIZEUS speech corpus. Results demonstrate that even in the presence of less number of speech samples available from the NOIZEUS speech corpus, the proposed metric outperforms in different contexts compared to the metric where the contexts are not classified before speech quality prediction.

Meta-reinforcement learning based few-shot speech reconstruction for non-intrusive speech quality assessment

Meta-reinforcement learning based few-shot speech reconstruction for non-intrusive speech quality assessment

Speech quality assessment with WARP‐Q: From similarity to subsequence dynamic time warp cost

Speech coding has been shown to achieve good speech quality using either waveform matching or parametric reconstruction. For very low bit rate streams, recently developed generative speech models can reconstruct high-quality wideband speech from the bit streams of standard parametric encoders at less than 3 kb/s. Generative codecs produce high-quality speech based on synthesising speech from a DNN and the parametric input. Existing objective speech quality models (e.g., ViSQOL and POLQA) cannot be used to accurately evaluate the quality of coded speech from generative models as they penalise based on signal differences not apparent in subjective listening test results. This paper presents WARP-Q, a full-reference objective speech quality metric that uses a dynamic time warping cost for MFCC representations of the signals. It is robust to low perceptual signal changes introduced by low bit rate neural vocoders. An evaluation using waveform matching, parametric, and generative neural vocoder-based codecs as well as channel and environmental noise shows that WARP-Q has better correlation and codec quality ranking for novel codecs compared to traditional metrics as well as the versatility of capturing other types of degradations, such as additive noise and transmission channel degradations.

Comparison of Speech Quality and Intelligibility Assessments in University Classrooms

Estimates of speech quality and intelligibility for three university classrooms of small, medium and large sizes are presented. The quality and intelligibility of speech were assessed by objective methods using binaural room impulse responses, measured at 5-6 points of the premises. The measures of speech quality were log-spectral distortion (LSD), bark spectral distortion (BSD) and perceptual evaluation of speech quality (PESQ), and the objective measure of speech intelligibility was the speech transmission index (STI). Among the quality measures considered, only BSD is shown to be highly correlated with STI measures for all three classrooms. In this case, correlation coefficient R varies from minus 0.6 for a small room to minus 0.98 for a large room. The close relationship between PESQ and STI is observed only in the case of a large classroom (R = 0.96-0.99), and the LSD measure was found to be uncorrelated with STI for premises of all sizes. The obtained results can serve as a justification for the use of BSD instead of STI, and vice versa, in the acoustic examination of classrooms of different sizes.