Non-intrusive Speech Quality Assessment Research Articles

ARCnet: A Multi-Feature-Based Auto Radio Check Model

Radio checks serve as the foundation for ground-to-air communication. To integrate machine learning for automated and reliable radio checks, this study introduces an Auto Radio Check network (ARCnet), a novel algorithm for non-intrusive speech quality assessment in civil aviation, addressing the crucial need for dependable ground-to-air communication. By employing a multi-scale feature fusion approach, including the consideration of audio’s frequency domain, comprehensibility, and temporal information within the radio check scoring network, ARCnet integrates manually designed features with self-supervised features and utilizes a transformer network to enhance speech segment analysis. Utilizing the NISQA open-source dataset and the proprietary RadioCheckSpeech dataset, ARCnet demonstrates superior performance in predicting speech quality, showing a 12% improvement in both the Pearson correlation coefficient and root mean square error (RMSE) compared to existing models. This research not only highlights the significance of applying multi-scale attributes and deep neural network parameters in speech quality assessment but also emphasizes the crucial role of the temporal network in capturing the nuances of voice data. Through a comprehensive comparison of the ARCnet approach to traditional methods, this study underscores its innovative contribution to enhancing communication efficiency and safety in civil aviation.

Non-intrusive speech quality assessment: A survey

Speech quality is a critical consideration for applications such as speech enhancement, coding, transmission, and synthesis. Accurately evaluating the quality of degraded speech without a reference is particularly challenging. As a result, non-intrusive speech quality assessment has been extensively researched due to its efficiency and ease of deployment. This survey comprehensively reviews these studies. Commonly used speech quality assessment databases are reviewed, as they serve as validation sets for objective metrics. Non-intrusive speech quality assessment methods are categorized and reviewed, and the performance of state-of-the-art and classical speech quality assessment methods is compared. The survey presents an overview of classic algorithms and recent advancements in the field of non-intrusive speech quality assessment.

Non-Intrusive Speech Quality Assessment Based on Deep Neural Networks for Speech Communication.

Traditionally, speech quality evaluation relies on subjective assessments or intrusive methods that require reference signals or additional equipment. However, over recent years, non-intrusive speech quality assessment has emerged as a promising alternative, capturing much attention from researchers and industry professionals. This article presents a deep learning-based method that exploits large-scale intrusive simulated data to improve the accuracy and generalization of non-intrusive methods. The major contributions of this article are as follows. First, it presents a data simulation method, which generates degraded speech signals and labels their speech quality with the perceptual objective listening quality assessment (POLQA). The generated data is proven to be useful for pretraining the deep learning models. Second, it proposes to apply an adversarial speaker classifier to reduce the impact of speaker-dependent information on speech quality evaluation. Third, an autoencoder-based deep learning scheme is proposed following the principle of representation learning and adversarial training (AT) methods, which is able to transfer the knowledge learned from a large amount of simulated speech data labeled by POLQA. With the help of discriminative representations extracted from the autoencoder, the prediction model can be trained well on a relatively small amount of speech data labeled through subjective listening tests. Fourth, an end-to-end speech quality evaluation neural network is developed, which takes magnitude and phase spectral features as its inputs. This phase-aware model is more accurate than the model using only the magnitude spectral features. A large number of experiments are carried out with three datasets: one simulated with labels obtained using POLQA and two recorded with labels obtained using subjective listening tests. The results show that the presented phase-aware method improves the performance of the baseline model and the proposed model with latent representations extracted from the adversarial autoencoder (AAE) outperforms the state-of-the-art objective quality assessment methods, reducing the root mean square error (RMSE) by 10.5% and 12.2% on the Beijing Institute of Technology (BIT) dataset and Tencent Corpus, respectively. The code and supplementary materials are available at https://github.com/liushenme/AAE-SQA.

Non-intrusive assessment of speech quality for classical Chinese poetry recitals using perceptual and acoustic features

Non-intrusive speech quality assessment is vital for objectively rating vast speech datasets without pristine reference signals. However, most existing research emphasizes speech clarity, neglecting tempo and rhythm. Different from normal Chinese speech, the prosody of classical Chinese poetry conveys profound emotion and a unique classical beauty. To assess the speech quality of classical Chinese poetry recitals across varied reciters and recording conditions, it's essential to design modules evaluating both clarity and prosody. In this paper, we propose a non-intrusive assessment method, utilizing perceptual and acoustic features as key indicators. First, by extracting the poetry's pitch frequency, we quantify the reference assessment function, deriving a prosodic score based on deviation degrees. Secondly, we devise a residual-structured neural network to capture the Mel-spectrogram's perceptual features, yielding the objective MOS score. Finally, we craft a comprehensive model using second-order polynomial regression to map both the objective MOS and prosody scores to the poetry recital quality score. After experimental validation, prosody and clarity modules both show outstanding prediction accuracy. The final quality assessment score performs a high correlation with human perception.

New research on monaural speech segregation based on quality assessment

Speech enhancement (SE) is a pivotal technology in enhancing the quality and intelligibility of speech signals. Nevertheless, when processing speech signals under conditions of high signal-to-noise ratio (SNR), conventional SE techniques may inadvertently lead to a diminution in the perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). This article introduces the innovative incorporation of the Non-Intrusive Speech Quality Assessment (NISQA) algorithm into SE systems. Through the comparison of pre and post-enhancement speech quality scores, it discerns whether the speech signal under consideration warrants enhancement processing, thereby mitigating potential deterioration in PESQ and STOI. Furthermore, this study delves into the ramifications of five prevalent speech features, namely, Mel Frequency Cepstral Coefficients (MFCC), Gammatone Frequency Cepstral Coefficients (GFCC), Relative Spectral Trans-formed Perceptual Linear Prediction coefficients (RASTA-PLP), Amplitude Modulation Spectrogram (AMS), and Multi-Resolution Cochleagram (MRCG), on PESQ and STOI under varying noise conditions. Experimental outcomes underscore that MRCG consistently emerges as the optimal and most stable feature for STOI, while the feature yielding the highest PESQ score exhibits intricate correlations with the background noise type, SNR level, and noise compatibility with the speech signal. Consequently, we propose an SE methodology founded on quality assessment and feature selection, facilitating the adaptive selection of optimal features tailored to distinct background noise scenarios, thereby always maintain the highest caliber enhancement effect with regard to PESQ metrics.

MSQAT: A multi-dimension non-intrusive speech quality assessment transformer utilizing self-supervised representations

Convolutional neural networks (CNNs) have been widely utilized as the main building block for many non-intrusive speech quality assessment (NISQA) methods. A new trend is to add a self-attention mechanism based on CNN to better capture long-term global content. However, it is not clear whether the pure attention-based network is sufficient to obtain good performance in NISQA. To this end, a framework named Multi-dimension non-intrusiveSpeech Quality Assessment Transformer (MSQAT) is proposed. To strengthen the interactions of various speech regions between local and global, we proposed the Audio Spectrogram Transformer Block (ASTB), Transposed Attention Block (TAB) and the Residual Swin Transformer Block (RSTB). These three modules employ attention mechanisms across spatial and channel dimensions, respectively. Additionally, speech quality varies not only in different frames, but also in different frequencies. Thus, a two-branch structure is designed to better evaluate the quality of speech by considering the weighting of each patch's score. Experimental results demonstrate that the proposed MSQAT has state-of-the-art performance on three standard datasets (NISQA Corpus, Tencent Corpus, and PSTN Corpus) and indicate that the pure attention model can achieve or surpass the performance of other CNN-attention hybrid models.

NISQE: Non-Intrusive Speech Quality Evaluator Based on Natural Statistics of Mean Subtracted Contrast Normalized Coefficients of Spectrogram

With the evolution in technology, communication based on the voice has gained importance in applications such as online conferencing, online meetings, voice-over internet protocol (VoIP), etc. Limiting factors such as environmental noise, encoding and decoding of the speech signal, and limitations of technology may degrade the quality of the speech signal. Therefore, there is a requirement for continuous quality assessment of the speech signal. Speech quality assessment (SQA) enables the system to automatically tune network parameters to improve speech quality. Furthermore, there are many speech transmitters and receivers that are used for voice processing including mobile devices and high-performance computers that can benefit from SQA. SQA plays a significant role in the evaluation of speech-processing systems. Non-intrusive speech quality assessment (NI-SQA) is a challenging task due to the unavailability of pristine speech signals in real-world scenarios. The success of NI-SQA techniques highly relies on the features used to assess speech quality. Various NI-SQA methods are available that extract features from speech signals in different domains, but they do not take into account the natural structure of the speech signals for assessment of speech quality. This work proposes a method for NI-SQA based on the natural structure of the speech signals that are approximated using the natural spectrogram statistical (NSS) properties derived from the speech signal spectrogram. The pristine version of the speech signal follows a structured natural pattern that is disrupted when distortion is introduced in the speech signal. The deviation of NSS properties between the pristine and distorted speech signals is utilized to predict speech quality. The proposed methodology shows better performance in comparison to state-of-the-art NI-SQA methods on the Centre for Speech Technology Voice Cloning Toolkit corpus (VCTK-Corpus) with a Spearman’s rank-ordered correlation constant (SRC) of 0.902, Pearson correlation constant (PCC) of 0.960, and root mean squared error (RMSE) of 0.206. Conversely, on the NOIZEUS-960 database, the proposed methodology shows an SRC of 0.958, PCC of 0.960, and RMSE of 0.114.

Non-intrusive speech quality assessment with attention-based ResNet-BiLSTM.

Speech quality is frequently affected by a variety factors in online conferencing applications, such as background noise, reverberation, packet loss and network jitter. In real scenarios, it is impossible to obtain a clean reference signal for evaluating the quality of the conferencing speech. Therefore, an effective non-intrusive speech quality assessment (NISQA) method is necessary. In this paper, we propose a new network framework for NISQA based on ResNet and BiLSTM. ResNet is utilized to extract local features, while BiLSTM is used to integrate representative features with long-term time dependencies and sequential characteristics. Considering that ResNet may result in the loss of context information when applied to the NISQA task, we propose a variant of ResNet which can preserve the time series information of the conferencing speech. The experimental results demonstrate that the proposed method has a high correlation with the mean opinion score of clean, noisy and processed speech.

Non-intrusive speech quality assessment using context-aware neural networks

To meet the human perceived quality of experience (QoE) while communicating over various Voice over Internet protocol (VoIP) applications, for example Google Meet, Microsoft Skype, Apple FaceTime, etc. a precise speech quality assessment metric is needed. The metric should be able to detect and segregate different types of noise degradations present in the surroundings before measuring and monitoring the quality of speech in real-time. Our research is motivated by the lack of clear evidence presenting speech quality metric that can firstly distinguish different types of noise degradations before providing speech quality prediction decision. To that end, this paper presents a novel non-intrusive speech quality assessment metric using context-aware neural networks in which the noise class (context) of the degraded or noisy speech signal is first identified using a classifier then deep neutral networks (DNNs) based speech quality metrics (SQMs) are trained and optimized for each noise class to obtain the noise class-specific (context-specific) optimized speech quality predictions (MOS scores). The noisy speech signals, that is, clean speech signals degraded by different types of background noises are taken from the NOIZEUS speech corpus. Results demonstrate that even in the presence of less number of speech samples available from the NOIZEUS speech corpus, the proposed metric outperforms in different contexts compared to the metric where the contexts are not classified before speech quality prediction.

Meta-reinforcement learning based few-shot speech reconstruction for non-intrusive speech quality assessment

Meta-reinforcement learning based few-shot speech reconstruction for non-intrusive speech quality assessment

Neural network-based non-intrusive speech quality assessment using attention pooling function

Recently, the non-intrusive speech quality assessment method has attracted a lot of attention since it does not require the original reference signals. At the same time, neural networks began to be applied to speech quality assessment and achieved good performance. To improve the performance of non-intrusive speech quality assessment, this paper proposes a neural network-based assessment method using attention pooling function. The proposed systems are based on the convolutional neural networks (CNNs), bidirectional long short-term memory (BLSTM), and CNN-LSTM structure. Comparing four types of pooling functions both theoretically and experimentally, we find the attention pooling function performs the best among the four. Experiments are conducted in a dataset containing various degraded speech signals with corresponding subjective quality scores. The results show that the proposed CNN-LSTM model using attention pooling function achieves state-of-the-art correlation coefficient (R) and root-mean-square error (RMSE) of 0.967 and 0.269, outperforming the performance of standardization ITU-T P.563 and autoencoder-support vector regression method.

Non-intrusive quality assessment of noise-suppressed speech using unsupervised deep features

Objective quality assessment aims towards evaluating the perceptual quality of a signal using a machine-based algorithm. Due to different challenges involved in the subjective evaluation of speech quality, it is necessary to develop objective measures. The goal of any non-intrusive quality assessment metric for noise-suppressed speech is to assess the quality of a noise-suppressed signal in the absence of any clean reference signal. As per the ITU-T P.835 recommendations, the quality assessment of noise-suppressed speech involves predicting three quality scores, namely, signal quality, background quality, and overall quality score, and hence, considered in this study. In recent literature, the non-intrusive quality assessment problem is presented as a regression problem, in which the mapping between a set of acoustic features and corresponding quality scores is found using a perceptual model. Recently, we proposed the use of Deep Autoencoder (DAE) features and Subband Autoencoder (SBAE) features for acoustic representation and an Artificial Neural Network (ANN) as a regression model. DAE and SBAE are variants of autoencoder architecture that have bottleneck structure in the hidden layers. Such architecture represents the class of generalized nonlinear Principal Component Analysis (PCA) that guarantees reconstruction of the input features with arbitrary accuracy. Both the features (DAE and SBAE) are extracted using unsupervised deep learning architectures, and they demonstrated better performance than the state-of-the-art spectral feature set, namely, Mel Filterbank Energies (FBEs). In this paper, we present more detailed analysis of previously proposed features, i.e., DAE and SBAE features, and analyze the usefulness of these features in predicting signal as well as background quality scores in addition to the overall quality score. We compare the performance of all the three features with each other as well as with current ITU-T P.563 metric for non-intrusive speech quality assessment. The results of our experiments performed on NOIZEUS database suggest that DAE and SBAE features perform relatively better than the FBEs while predicting signal and overall quality. On the other hand, FBE features perform slightly better than the DAE and SBAE features in predicting the background quality. Moreover, another major contribution of this paper is that we employ an ANN to predict all the three quality scores simultaneously, and present the results. It was observed that using this approach, it is possible to predict all the three scores simultaneously with similar accuracy as that of predicting them individually.

Speech Quality Assessment in Wireless VoIP Communication Using Deep Belief Network

Nowadays, the voice over Internet protocol (VoIP) communication service is widely adopted, and it counts with many users across the world. However, the users’ quality of experience is not guaranteed because the voice signal quality can be affected by several degradations that happen in the network infrastructure. Thus, it is relevant to have a global speech quality assessment method that considers both wired and wireless networks to provide reliable results. In this paper, several network scenarios that consider different packet loss rates (PLRs) and wireless channel models are implemented in which the impaired signals are evaluated using the algorithm described in ITU-T Recommendation P.862. Preliminary results showed a relationship between both fading and PLR parameters and the global speech quality index. However, the P.862 algorithm is not viable in real VoIP scenarios. The ITU-T Recommendation P.563 describes a non-intrusive speech quality assessment method; nevertheless, its results are not confident. In this context, the main objective of this paper is to propose a non-intrusive speech quality classification model based on a deep belief network (DBN) that considers the wired and wireless impairments on the speech signal. Experimental results demonstrated a high correlation between the proposed model based on the DBN and P.862 algorithm, reaching a ${F}$ -measure of 97.01%. For validation, the non-intrusive P.563 algorithm is used; the proposed model and P.563 reached an average accuracy of 96.14% and 72.12%, respectively. Furthermore, subjective tests were carried out, and the proposed DBN model reached an accuracy of 94%.

Sparse representation‐based quasi‐clean speech construction for speech quality assessment under complex environments

A non-intrusive speech quality assessment method for complex environments was proposed. In the proposed approach, a new sparse representation-based speech reconstruction algorithm was presented to acquire the quasi-clean speech from the noisy degraded signal. Firstly, an over-complete dictionary of the clean speech power spectrum was learned by the K-singular value decomposition algorithm. Then in the sparse representation stage, the stopping residue error was adaptively achieved according to the estimated cross-correlation and the noise spectrum which was adjusted by a posteriori SNR-weighted factor, and the orthogonal matching pursuit approach was applied to reconstruct the clean speech spectrum from the noisy speech. The quasi-clean speech was considered as the reference to a modified PESQ perceptual model, and the mean opinion score of the noisy degraded speech was achieved via the distortions estimation between the quasi-clean speech and the degraded speech. Experimental results show that the proposed approach obtains a correlation coefficient of 0.925 on NOIZEUS complex environment database, which is 99% similar to the performance of the intrusive standard ITU-T PESQ, and 7.1% outperforms non-intrusive standard ITU-T P.563.

E-Model Modification for Multi-Languages over IP

It is well known that non-intrusive speech quality assessment methods are appropriate for real time monitoring of VoIP traffic. However, previous researches has proved that most of the non-intrusive speech quality assessment methods failed to estimate accurate speech quality using different languages. Consequently, intrusive methods are frequently chosen to provide a more accurate measurement, however they cannot be used for real time VoIP traffic monitoring. In this paper, the technique to enhanced simplified version of ITU-T recommendation G.107 E-model with a language impairment parameter was proposed. The method to estimate the function of language impairment by tuning the E-model with an intrusive objective method, PESQ was presented. The results from statistical analysis show that the modified E-model matches well with PESQ scores in eight languages.DOI: http://dx.doi.org/10.5755/j01.eee.21.1.7612

Non-intrusive speech quality assessment using several combinations of auditory features

Quality estimation of speech is essential for monitoring and maintenance of the quality of service at different nodes of modern telecommunication networks. It is also required in the selection of codecs in speech communication systems. There is no requirement of the original clean speech signal as a reference in non-intrusive speech quality evaluation, and thus it is of importance in evaluating the quality of speech at any node of the communication network. In this paper, non-intrusive speech quality assessment of narrowband speech is done by Gaussian Mixture Model (GMM) training using several combinations of auditory perception and speech production features, which include principal components of Lyon's auditory model features, MFCC, LSF and their first and second differences. Results are obtained and compared for several combinations of auditory features for three sets of databases. The results are also compared with ITU-T Recommendation P.563 for non-intrusive speech quality assessment. It is found that many combinations of these feature sets outperform the ITU-T P.563 Recommendation under the test conditions.

Non-intrusive Speech Quality Assessment in VoIP Using the Extended E-model

Non-intrusive Speech Quality Assessment in VoIP Using the Extended E-model

Low-Complexity, Nonintrusive Speech Quality Assessment

Monitoring of speech quality in emerging heterogeneous networks is of great interest to network operators. The most efficient way to satisfy such a need is through nonintrusive, objective speech quality assessment. In this paper, we describe a low-complexity algorithm for monitoring the speech quality over a network. The features used in the proposed algorithm can be computed from commonly used speech-coding parameters. Reconstruction and perceptual transformation of the signal is not performed. The critical advantage of the approach lies in generating quality assessment ratings without explicit distortion modeling. The results from the performed experiments indicate that the proposed nonintrusive objective quality measure performs better than the ITU-T P.563 standard

Non-intrusive speech-quality assessment using vocal-tract models

It is increasingly vital that there are effective quality of service (QoS) metrics to describe the performance of telecommunications networks. Speech quality is a major contributor to users' perception of QoS, and the ability to design for and monitor this quality is paramount. The authors describe work towards a non-intrusive speech quality assessment algorithm, capable of making predictions of the speech quality received by a customer, utilising the in-service signal. Modern telecommunications networks contain complex nonlinear elements that cannot be assessed with traditional engineering metrics. A novel use of vocal-tract modelling techniques is described, which enables predictions of the quality of a network degraded speech stream to be made. Details of the algorithm's adaptation to different talker characteristics are presented, together with a summary of the performance of the system.