Evaluation Of Speech Quality Research Articles

A Non-Invasive Speech Quality Evaluation Algorithm for Hearing Aids With Multi-Head Self-Attention and Audiogram-Based Features

A Non-Invasive Speech Quality Evaluation Algorithm for Hearing Aids With Multi-Head Self-Attention and Audiogram-Based Features

Speech Intelligibility and Quality Evaluation of Automotive Microphones Using Different Test Metrics and Their Correlation

Speech Intelligibility and Quality Evaluation of Automotive Microphones Using Different Test Metrics and Their Correlation

New research on monaural speech segregation based on quality assessment

Speech enhancement (SE) is a pivotal technology in enhancing the quality and intelligibility of speech signals. Nevertheless, when processing speech signals under conditions of high signal-to-noise ratio (SNR), conventional SE techniques may inadvertently lead to a diminution in the perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI). This article introduces the innovative incorporation of the Non-Intrusive Speech Quality Assessment (NISQA) algorithm into SE systems. Through the comparison of pre and post-enhancement speech quality scores, it discerns whether the speech signal under consideration warrants enhancement processing, thereby mitigating potential deterioration in PESQ and STOI. Furthermore, this study delves into the ramifications of five prevalent speech features, namely, Mel Frequency Cepstral Coefficients (MFCC), Gammatone Frequency Cepstral Coefficients (GFCC), Relative Spectral Trans-formed Perceptual Linear Prediction coefficients (RASTA-PLP), Amplitude Modulation Spectrogram (AMS), and Multi-Resolution Cochleagram (MRCG), on PESQ and STOI under varying noise conditions. Experimental outcomes underscore that MRCG consistently emerges as the optimal and most stable feature for STOI, while the feature yielding the highest PESQ score exhibits intricate correlations with the background noise type, SNR level, and noise compatibility with the speech signal. Consequently, we propose an SE methodology founded on quality assessment and feature selection, facilitating the adaptive selection of optimal features tailored to distinct background noise scenarios, thereby always maintain the highest caliber enhancement effect with regard to PESQ metrics.

In-car speech enhancement based on source separation technique

This work addresses the solution of localizing and enhancing hands-free speech inside the car environment. Cars have different types of sounds from outside, co-passengers dialogue and noise. To provide better-quality speech, a microphone array-based beamforming technique is used. This research work proposes the method for selected source localization, source separation, and enhancement. An estimation of the direction of arrival (DOA) to localize the signal direction and preferred direction is selected for speech enhancement. The spiral and sine-cosine algorithm (SSCA) algorithm is combined with an adaptive least mean square to adapt the system for different environments. The algorithm is implemented in hardware and tested in a real-time car environment. The results showed significant improvement in signal-to-noise ratio (SNR) of 5.2 dB and perceptual evaluation of speech quality (PESQ) of 2.3. Finally, the model is fine-tuned for the car to get better quality. The proposed technique is efficient, and results are compared with existing methods.

Noise estimation based on optimal smoothing and minimum controlled through recursive averaging for speech enhancement

One of the most significant challenges in the real time speech processing applications is the elimination of noise in the corrupted speech data. This noise can significantly impact the efficacy/performance of the applications of speech processing. However, developing a robust noise reduction algorithm is crucial for improving the accuracy of automatic speech recognition (ASR) and other speech processing systems under uncontrolled conditions. In this paper, we propose an algorithm to reduce the background noise in the degraded speech data under highly non-stationary conditions. The proposed optimal smoothing and minima controlled (OSMC) technique uses recursive averaging to enhance degraded speech data. Initially, a smoothed periodogram and local minima of the degraded speech data are computed and determined the time-frequency dependent threshold factor. The ratio of smoothed periodogram to local minima is used to find the active regions of speech in the degraded speech data by adapting the Bayesian minimum cost decision rule. To calculate the estimated noise spectrum for each frequency bin, a time-frequency smoothing factors are used. The perceptual evaluation of speech quality (PESQ) and normalized covariance metric (NCM) are used to evaluate the speech quality and intelligibility of the proposed technique over competing algorithms after speech enhancement. The experimental results demonstrated that the proposed algorithm has given a significant improvement in terms of average values of PESQ by 15.03% and 16.71% and NCM by 3.45% and 5.73% for NOIZEUS and Kannada speech databases at 5 dB and 10 dB respectively, over unprocessed speech under highly non-stationary noisy environments.

A robust speech enhancement method in noisy environments

Speech enhancement aims to eliminate or reduce undesirable noises and distortions, this processing should keep features of the speech to enhance the quality and intelligibility of degraded speech signals. In this study, we investigated a combined approach using single-frequency filtering (SFF) and a modified spectral subtraction method to enhance single-channel speech. The SFF method involves dividing the speech signal into uniform subband envelopes, and then performing spectral over-subtraction on each envelope. A smoothing parameter, determined by the a-posteriori signal-to-noise ratio (SNR), is used to estimate and update the noise without the need for explicitly detecting silence. To evaluate the performance of our algorithm, we employed objective measures such as segmental SNR (segSNR), extended short-term objective intelligibility (ESTOI), and perceptual evaluation of speech quality (PESQ). We tested our algorithm with various types of noise at different SNR levels and achieved results ranging from 4.24 to 15.41 for segSNR, 0.57 to 0.97 for ESTOI, and 2.18 to 4.45 for PESQ. Compared to other standard and existing speech enhancement methods, our algorithm produces better results and performs well in reducing undesirable noises.

Bidirectional EMD-RLS: Performance analysis for denoising in speech signal

Empirical Mode Decomposition (EMD) is an alternative decomposition method that allows the analysis of non-stationary and non-linear signals, decomposing them into components called Intrinsic Mode Functions (IMFs). In this paper, we introduce the bidirectional combination between EMD and the Recursive Least Squares (RLS) adaptive filter and compare it to Least Mean Square (LMS), EMD-LMS and RLS methods. The hybrid algorithm relies on the determination of vertical (IMFs indexes) and horizontal (temporal indexes) weighting coefficients. We evaluated the effectiveness of the methods by filtering sets of speech signals artificially corrupted by white Gaussian noise. To find the method that provides the greatest noise reduction and the best speech quality, we took into account the Signal-to-Noise Ratio and the Perceptual Evaluation of Speech Quality (PESQ) score. The analyzed cases showed that EMD-RLS can provide results as good as the other methods, but with a shorter learning curve than LMS and EMD-LMS, despite the higher computational cost. For a single training iteration, the methods presented PESQ scores on average equivalent to each other. However, they showed a significant difference in performance regarding noise reduction, with the best result obtained by RLS, followed by EMD-RLS, EMD-LMS and LMS.

Non-Intrusive Air Traffic Control Speech Quality Assessment with ResNet-BiLSTM

In the current field of air traffic control speech, there is a lack of effective objective speech quality evaluation methods. This paper proposes a new network framework based on ResNet–BiLSTM to address this issue. Firstly, the mel-spectrogram of the speech signal is segmented using the sliding window technique. Next, a preceding feature extractor composed of convolutional and pooling layers is employed to extract shallow features from the mel-spectrogram segment. Then, ResNet is utilized to extract spatial features from the shallow features, while BiLSTM is used to extract temporal features, and these features are horizontally concatenated. Finally, based on the concatenated spatiotemporal features, the final speech quality score is computed using fully connected layers. We conduct experiments on the air traffic control speech database and compare the objective scoring results with the subjective scoring results. The experimental results demonstrate that the proposed method has a high correlation with the mean opinion score (MOS) of air traffic control speech.

Cascade algorithms for combined acoustic feedback cancelation and noise reduction

This paper presents three cascade algorithms for combined acoustic feedback cancelation (AFC) and noise reduction (NR) in speech applications. A prediction error method (PEM)-based adaptive feedback cancelation (PEM-based AFC) algorithm is used for the AFC stage, while a multichannel Wiener filter (MWF) is applied for the NR stage. A scenario with M microphones and 1 loudspeaker is considered, without loss of generality. The first algorithm is the baseline algorithm, namely the cascade M-channel rank-1 MWF and PEM-AFC, where a NR stage is performed first using a rank-1 MWF followed by a single-channel AFC stage using a PEM-based AFC algorithm. The second algorithm is the cascade (M+1)-channel rank-2 MWF and PEM-AFC, where again a NR stage is applied first followed by a single-channel AFC stage. The novelty of this algorithm is to consider an (M+1)-channel data model in the MWF formulation with two different desired signals, i.e., the speech component in the reference microphone signal and in the loudspeaker signal, both defined by the speech source signal but not equal to each other. The two desired signal estimates are later used in a single-channel PEM-based AFC stage. The third algorithm is the cascade M-channel PEM-AFC and rank-1 MWF where an M-channel AFC stage is performed first followed by an M-channel NR stage. Although in cascade algorithms where NR is performed first and then AFC the estimation of the feedback path is usually affected by the NR stage, it is shown here that by performing a rank-2 approximation of the speech correlation matrix this issue can be avoided and the feedback path can be correctly estimated. The performance of the algorithms is assessed by means of closed-loop simulations where it is shown that for the considered input signal-to-noise ratios (iSNRs) the cascade (M+1)-channel rank-2 MWF and PEM-AFC and the cascade M-channel PEM-AFC and rank-1 MWF algorithms outperform the cascade M-channel rank-1 MWF and PEM-AFC algorithm in terms of the added stable gain (ASG) and misadjustment (Mis) as well as in terms of perceptual metrics such as the short-time objective intelligibility (STOI), perceptual evaluation of speech quality (PESQ), and signal distortion (SD).

Enhancements in encoded noisy speech data by background noise reduction

In this paper, we present recent improvements incorporated into an earlier speech encoding system that combined spectral subtraction with voice activity detection (SS-VAD) and linear predictive coding (LPC). The presence of background noise in the speech data reduces the speech audibility, quality, and intelligibility. In the previously proposed system, musical and other types of noises had an adverse effect on encoding performance under low signal-to-noise ratio (SNR) conditions. To address this issue, we propose a technique by amalgamating the minimum mean square error-spectrum power estimator based on the zero-crossing (MMSE-SPZC) and LPC for the development of a noise robust encoding system. The performance evaluation is done for noisy and enhanced encoded speech data using perceptual evaluation of speech quality (PESQ), composite measures (CM), and the normalized covariance metric (NCM). The newly proposed encoding system has provided better PESQ, CM, and NCM values for various types of noises at different SNR levels compared to the earlier reported SS-VAD and LPC-based system. In addition to performance evaluation, we also present the complexity of the computation and memory requirements for both the proposed and existing algorithms. The source code of the algorithms used in this work are made publicly available.1

A Smart Control System for the Oil Industry Using Text-to-Speech Synthesis Based on IIoT

Oil refineries have high operating expenses and are often exposed to increased asset integrity risks and functional failure. Real-time monitoring of their operations has always been critical to ensuring safety and efficiency. We proposed a novel Industrial Internet of Things (IIoT) design that employs a text-to-speech synthesizer (TTS) based on neural networks to build an intelligent extension control system. We enhanced a TTS model to achieve high inference speed by employing HiFi-GAN V3 vocoder in the acoustic model FastSpeech 2. We experimented with our system on a low resources-embedded system in a real-time environment. Moreover, we customized the TTS model to generate two target speakers (female and male) using a small dataset. We performed an ablation analysis by conducting experiments to evaluate the performance of our design (IoT connectivity, memory usage, inference speed, and output speech quality). The results demonstrated that our system Real-Time Factor (RTF) is 6.4 (without deploying the cache mechanism, which is a technique to call the previously synthesized speech sentences in our system memory). Using the cache mechanism, our proposed model successfully runs on a low-resource computational device with real-time speed (RTF equals 0.16, 0.19, and 0.29 when the memory has 250, 500, and 1000 WAV files, respectively). Additionally, applying the cache mechanism has reduced memory usage percentage from 16.3% (for synthesizing a sentence of ten seconds) to 6.3%. Furthermore, according to the objective speech quality evaluation, our TTS model is superior to the baseline TTS model.

Comparison of objective and subjective methods for evaluating speech quality and intelligibility recorded through bone conduction and in-ear microphones

Bone conduction and in-ear microphones pick up the Bone-conducted (BC) speech, which has traveled through bones and soft tissues. This BC speech is less sensitive to surrounding noise than Air-conducted (AC) speech. Therefore, there is an interest in recording this signal. However, the intelligibility and quality of these microphones are known to be a limit to their use. Previous work has noted confusion between vowel sounds in terms of intelligibility. To evaluate the quality and intelligibility, studies rely on subjective and objective tests. This paper aims to determine if the standard objective methods for rating speech quality and intelligibility can be applied to BC speech recorded through these microphones. In order to estimate intelligibility, a subjective test based on vowel recognition was compared to STOI and to a new criterion based on second formant frequencies of oral vowels. For speech quality estimation, MUSHRA and PESQ tests were compared. Results show difficulties in using objective methods instead of subjective ones. Therefore, for ongoing studies, it is suggested to use subjective methods to evaluate speech quality and intelligibility.

Speech Enhancement Based on Enhanced Empirical Wavelet Transform and Teager Energy Operator

This paper presents a new speech-enhancement approach based on an enhanced empirical wavelet transform, considering the time and scale adaptation of thresholds for individual component signals obtained from the used transform. The time adaptation is performed using the Teager energy operator on the individual component signals, and the scale adaptation of thresholds is performed by the modified level-dependent threshold principle for the individual component signals. The proposed approach does not require an explicit estimation of the noise level or a priori knowledge of the signal-to-noise ratio as is usually needed in most common speech-enhancement methods. The effectiveness of the proposed method has been assessed based on over 1000 speech recordings from the public Librispeech database. The research included various types of noise (among others white, violet, brown, blue, and pink) and various types of disturbance (among others traffic sounds, hair dryer, and fan), which were added to the selected test signals. The score of perceptual evaluation of speech quality, allowing for the assessment of the quality of enhanced speech, and signal-to-noise ratio, allowing for the assessment of the effectiveness of disturbance attenuation, are selected for the evaluation of the resultant effectiveness of the proposed approach. The resultant effectiveness of the proposed approach is compared with other selected speech-enhancement methods or denoising techniques available in the literature. The experimental research results show that the proposed method performs better than conventional methods in many types of high-noise conditions in terms of producing less residual noise and lower speech distortion.

Sparse Signal Recovery through Long Short-Term Memory Networks for Compressive Sensing-Based Speech Enhancement

This paper presents a novel speech enhancement approach based on compressive sensing (CS) which uses long short-term memory (LSTM) networks for the simultaneous recovery and enhancement of the compressed speech signals. The advantage of this algorithm is that it does not require an iterative process to recover the compressed signals, which makes the recovery process fast and straight forward. Furthermore, the proposed approach does not require prior knowledge of signal and noise statistical properties for sensing matrix optimization because the used LSTM can directly extract and learn the required information from the training data. The proposed technique is evaluated against white, babble, and f-16 noises. To validate the effectiveness of the proposed approach, perceptual evaluation of speech quality (PESQ), short-time objective intelligibility (STOI), and signal-to-distortion ratio (SDR) were compared to other variants of OMP-based CS algorithms The experimental outcomes show that the proposed approach achieves the maximum improvements of 50.06%, 43.65%, and 374.16% for PESQ, STOI, and SDR respectively, over the different variants of OMP-based CS algorithms.

Chinese Speech Enhancement and Adaptive Recognition Technology for Complex Language Environments

The development of intelligent technology has also made rapid progress in relevant speech fields. In order to increase the application scenarios of speech recognition systems, the research has improved the traditional Speech enhancement algorithm, namely the Ideal Binary Mask (IBM) algorithm, and combined it with the unimproved IBM algorithm to propose an adaptive IBM algorithm. Based on this algorithm, the research has built a new speech recognition system, The system uses an FIR filter to realize pre-emphasis processing and uses Berouti spectral subtraction to preprocess speech. The Speech enhancement model is built using a deep learning network model. The results showed that the IBM algorithm had the highest score in the Perceptual Evaluation of Speech Quality (PESQ) at 3.5596, followed by the Ideal Ratio Mask (IRM) algorithm at 3.3429. The improvement of the IBM algorithm was feasible when the noise intensity coefficient was greater than 0.008. When the noise intensity coefficient was greater than 0.08, the average score of the improved IBM algorithm was 2.1079, and the average score of the unimproved IBM algorithm was 1.9418. The proposed adaptive IBM algorithm has higher performance in complex speech environments compared to the original system.

Self-Attention-Based Convolutional GRU for Enhancement of Adversarial Speech Examples

Recent research has identified adversarial examples which are the challenges to DNN-based ASR systems. In this paper, we propose a new model based on Convolutional GRU and Self-attention U-Net called [Formula: see text] to improve adversarial speech signals. To represent the correlation between neighboring noisy speech frames, a two-Layer GRU is added in the bottleneck of U-Net and an attention gate is inserted in up-sampling units to increase the adversarial stability. The goal of using GRU is to combine the weights sharing technique with the use of gates to control the flow of data across multiple feature maps. As a result, it outperforms the original 1D convolution used in [Formula: see text]. Especially, the performance of the model is evaluated by explainable speech recognition metrics and its performance is analyzed by the improved adversarial training. We used adversarial audio attacks to perform experiments on automatic speech recognition (ASR). We saw (i) the robustness of ASR models which are based on DNN can be improved using the temporal features grasped by the attention-based GRU network; (ii) through adversarial training, including some additive adversarial data augmentation, we could improve the generalization power of Automatic Speech Recognition models which are based on DNN. The word-error-rate (WER) metric confirmed that the enhancement capabilities are better than the state-of-the-art [Formula: see text]. The reason for this enhancement is the ability of GRU units to extract global information within the feature maps. Based on the conducted experiments, the proposed [Formula: see text] increases the score of Speech Transmission Index (STI), Perceptual Evaluation of Speech Quality (PESQ), and the Short-term Objective Intelligibility (STOI) with adversarial speech examples in speech enhancement.

Effects of the Number of Channels and Channel Stimulation Rate on Speech Recognition and Sound Quality Using Precurved Electrode Arrays.

This study investigated the relationship between the number of active electrodes, channel stimulation rate, and their interaction on speech recognition and sound quality measures while controlling for electrode placement. Cochlear implant (CI) recipients with precurved electrode arrays placed entirely within scala tympani and closer to the modiolus were hypothesized to be able to utilize more channels and possibly higher stimulation rates to achieve better speech recognition performance and sound quality ratings than recipients in previous studies. Participants included seven postlingually deafened adult CI recipients with Advanced Bionics Mid-Scala electrode arrays confirmed to be entirely within scala tympani using postoperative computerized tomography. Twelve conditions were tested using four, eight, 12, and 16 electrodes and channel stimulation rates of 600 pulse per second (pps), 1,200 pps, and each participant's maximum allowable rate (1,245-4,800 pps). Measures of speech recognition and sound quality were acutely assessed. For the effect of channels, results showed no significant improvements beyond eight channels for all measures. For the effect of channel stimulation rate, results showed no significant improvements with higher rates, suggesting that 600 pps was sufficient for maximum speech recognition performance and sound quality ratings. However, across all conditions, there was a significant relationship between mean electrode-to-modiolus distance and all measures, suggesting that a lower mean electrode-to-modiolus distance was correlated with higher speech recognition scores and sound quality ratings. These findings suggest that even well-placed precurved electrode array recipients may not be able to take advantage of more than eight channels or higher channel stimulation rates (> 600 pps), but that closer electrode array placement to the modiolus correlates with better outcomes for these recipients.

On the deficiency of intelligibility metrics as proxies for subjective intelligibility

A recent trend in deep neural network (DNN)-based speech enhancement consists of using intelligibility and quality metrics as loss functions for model training with the aim of achieving high subjective speech intelligibility and perceptual quality in real-life conditions. In this study, we analyze a variety of loss functions, including some based on state-of-the-art intelligibility and quality metrics, to train an end-to-end speech enhancement system based on a fully convolutional neural network. The loss functions include perceptual metric for speech quality evaluation (PMSQE), scale-invariant signal-to-distortion ratio (SI-SDR), SI-SDR integrating speech pre-emphasis, short-time objective intelligibility (STOI), extended STOI (ESTOI), spectro-temporal glimpsing index (STGI), and a composite loss function combining STGI and SI-SDR. While DNNs trained with these loss functions produce notable speech intelligibility (and quality) gains according to pertinent objective metrics, we conduct a subjective intelligibility test that contradicts this result, showing no intelligibility improvement. From the results of this study, our conclusion is twofold: (1) subjective intelligibility evaluation is currently not replaceable by objective intelligibility evaluation, and (2) both the development of meaningful intelligibility metrics and DNN-based speech enhancement systems that can consistently improve the intelligibility of noisy speech for human listening remain open problems.

Numerical Ratings and Content Labeling of Speeches in an Educational Public Speaking Program

<p style="text-align: justify;">Education in public speaking often relies largely on qualitative feedback. In this study, we explored how a numerical feedback form based on a validated scale could be used in an educational public speaking program. Data were collected at three clubs of Toastmasters International. Speech content was labeled to enable statistical analysis of correlations between speech content and audience ratings. 216 speeches by 59 speakers were evaluated by audience members, providing 1416 individual evaluations. All rating categories correlated strongly with each other. Speeches mentioning relationships, personal stories and positive emotions were rated more favorably. Speeches given in-person were rated more highly in several dimensions compared to speeches given through online video. There were some sex differences in choice of content and emotional expressiveness, but not in audience ratings of speech quality. Overall, the study found that the method was useful for both providing individual feedback and aggregated data for research purposes.</p>

Bluetooth Coupling in Hearing Aids: Effect on Audiovisual Speech Recognition and Quality Rating of Compressed Speech in Older Individuals with Sloping Hearing Loss.

Introduction Older individuals often report that they find it difficulty in enjoying watching television since they find it hard to follow the rapid intensity variations, and voice changes from scene to scene. Objective The present study investigated the effect of coupling the hearing aid with the television via Bluetooth on audiovisual speech recognition and quality rating of compressed speech in older individuals with hearing loss. Method Twenty participants in the age range of 60 to 75 years who had moderate to moderately severe sloping sensorineural hearing loss were bilaterally fitted with digital receiver in the canal hearing aids. The hearing aid was coupled with a television via Bluetooth using a streamer. The video recorded stimuli were presented at 65 dB SPL at normal rate, 35% compression and 45% compression conditions. Speech recognition scores and quality ratings were obtained for each condition with and without the Bluetooth streamer connected to the hearing aids. Results Speech recognition scores were significantly better with Bluetooth coupling compared with conventional hearing aid use at 40% compressed speech rate. The quality was also rated higher in almost all parameters across speech rates when Bluetooth was used. Conclusions The improved clarity and nullification of room reverberation offered by Bluetooth coupling can potentially compensate for the age-related temporal processing deficit contributing to ease of listening.