Subjective Listening Tests Research Articles

Phase perception of the glottal excitation and its relevance in statistical parametric speech synthesis

While the characteristics of the amplitude spectrum of the voiced excitation have been studied widely both in natural and synthetic speech, the role of the excitation phase has remained less explored. This contradicts findings observed in sound perception studies indicating that humans are not phase deaf. Especially in speech synthesis, phase information is often omitted for simplicity. This study investigates the impact of phase information of the excitation signal of voiced speech and its relevance in statistical parametric speech synthesis. The experiments in the study involve, firstly, converting the pitch-synchronously computed original phase spectra of the excitation waveforms (either glottal flow waveforms or residuals) to either zero phase, cyclostationary random phase, or random phase. Secondly, the quality of synthetic speech in each case is compared in subjective listening tests to the corresponding signal excited with the original, natural phase. Experiments are conducted with natural, vocoded, and synthetic speech using voice material from various speakers with varying speaking styles, such as breathy, normal, and Lombard speech. The results indicate that the phase spectrum of the voiced excitation has a perceptually relevant effect in natural, vocoded, and synthetic speech, and utilizing the phase information in speech synthesis leads to improved speech quality.

Array model interpolation and subband iterative adaptive filters applied to beamforming-based acoustic echo cancellation.

In this paper, an evolutionary exposition is given in regard to the enhancing strategies for acoustic echo cancellers (AECs). A fixed beamformer (FBF) is utilized to focus on the near-end speaker while suppressing the echo from the far end. In reality, the array steering vector could differ considerably from the ideal freefield plane wave model. Therefore, an experimental procedure is developed to interpolate a practical array model from the measured frequency responses. Subband (SB) filtering with polyphase implementation is exploited to accelerate the cancellation process. Generalized sidelobe canceller (GSC) composed of an FBF and an adaptive blocking module is combined with AEC to maximize cancellation performance. Another enhancement is an internal iteration (IIT) procedure that enables efficient convergence in the adaptive SB filters within a sample time. Objective tests in terms of echo return loss enhancement (ERLE), perceptual evaluation of speech quality (PESQ), word recognition rate for automatic speech recognition (ASR), and subjective listening tests are conducted to validate the proposed AEC approaches. The results show that the GSC-SB-AEC-IIT approach has attained the highest ERLE without speech quality degradation, even in double-talk scenarios.

High quality voice conversion using prosodic and high-resolution spectral features

Voice conversion methods have advanced rapidly over the last decade. Studies have shown that speaker characteristics are captured by spectral feature as well as various prosodic features. Most existing conversion methods focus on the spectral feature as it directly represents the timbre characteristics, while some conversion methods have focused only on the prosodic feature represented by the fundamental frequency. In this paper, a comprehensive framework using deep neural networks to convert both timbre and prosodic features is proposed. The timbre feature is represented by a high-resolution spectral feature. The prosodic features include F0, intensity and duration. It is well known that DNN is useful as a tool to model high-dimensional features. In this work, we show that DNN initialized by our proposed autoencoder pretraining yields good quality DNN conversion models. This pretraining is tailor-made for voice conversion and leverages on autoencoder to capture the generic spectral shape of source speech. Additionally, our framework uses segmental DNN models to capture the evolution of the prosodic features over time. To reconstruct the converted speech, the spectral feature produced by the DNN model is combined with the three prosodic features produced by the DNN segmental models. Our experimental results show that the application of both prosodic and high-resolution spectral features leads to quality converted speech as measured by objective evaluation and subjective listening tests.

Encoding and communicating navigable speech soundfields

This paper describes a system for encoding and communicating navigable speech soundfields for applications such as immersive audio/visual conferencing, audio surveillance of large spaces and free viewpoint television. The system relies on recording speech soundfields using compact co-incident microphone arrays that are then processed to identify sources and their spatial location using the well-known assumption that speech signals are sparse in the time-frequency domain. A low-delay Direction of Arrival (DOA)-based frequency domain sound source separation approach is proposed that requires only 250 ms of speech signal. Joint compression is achieved through a previously proposed perceptual analysis-by-synthesis spatial audio coding scheme that encodes sources into a mixture signal that can be compressed by a standard speech codec at 32 kbps. By also transmitting side information representing the original spatial location of each source, the received mixtures can be decoded and then flexibly reproduced using loudspeakers at a chosen listening point within a synthesised speech scene. The system was implemented based on this framework for an example application encoding a three-talker navigable speech scene at a total bit rate of 48 kbps. Subjective listening tests were conducted to evaluate the quality of the reproduced speech scenes at a new listening point as compared to a true recording at that point. Results demonstrate the approach successfully encodes multiple spatial speech scenes at low bit rates whilst maintaining perceptual quality in both anechoic and reverberant environments.

Generalized maximum a posteriori spectral amplitude estimation for speech enhancement

Spectral restoration methods for speech enhancement aim to remove noise components in noisy speech signals by using a gain function in the spectral domain. How to design the gain function is one of the most important parts for obtaining enhanced speech with good quality. In most studies, the gain function is designed by optimizing a criterion based on some assumptions of the noise and speech distributions, such as minimum mean square error (MMSE), maximum likelihood (ML), and maximum a posteriori (MAP) criteria. The MAP criterion shows advantage in obtaining a more reliable gain function by incorporating a suitable prior density. However, it has a problem as several studies showed: although MAP based estimator effectively reduces noise components when the signal-to-noise ratio (SNR) is low, it brings large speech distortion when the SNR is high. For solving this problem, we have proposed a generalized maximum a posteriori spectral amplitude (GMAPA) algorithm in designing a gain function for speech enhancement. The proposed GMAPA algorithm dynamically specifies the weight of prior density of speech spectra according to the SNR of the testing speech signals to calculate the optimal gain function. When the SNR is high, GMAPA adopts a small weight to prevent overcompensations that may result in speech distortions. On the other hand, when the SNR is low, GMAPA uses a large weight to avoid disturbance of the restoration caused by measurement noises. In our previous study, it has been proven that the weight of the prior density plays a crucial role to the GMAPA performance, and the weight is determined based on the SNR in an utterance-level. In this paper, we propose to compute the weight with the consideration of time–frequency correlations that result in a more accurate estimation of the gain function. Experiments were carried out to evaluate the proposed algorithm on both objective tests and subjective tests. The experimental results obtained from objective tests indicate that GMAPA is promising compared to several well-known algorithms at both high and low SNRs. The results of subjective listening tests indicate that GMAPA provides significantly higher sound quality than other speech enhancement algorithms.

Reduction of Gaussian, Supergaussian, and Impulsive Noise by Interpolation of the Binary Mask Residual

In this paper, we present a new approach for noise reduction. A binary time-frequency (T-F) masking threshold criterion is proposed and analyzed with respect to the average spectra of music and noise disturbances. Modified autoregressive (AR) detection and AR interpolation are then applied to the residual signal of the binary masking process. The proposed method is able to reduce supergaussian and impulsive noise while ensuring preservation of the desired signal, which is crucial for professional high-quality audio restoration, and it is also suitable for Gaussian noise to a certain extent. The approach is compared to a state-of-the-art restoration algorithm by means of the objective measures signal-to-noise ratio (SNR) improvement and perceptual quality, and by subjective listening tests. The objective results as well as the listening tests show that the proposed algorithm is especially suited for supergaussian, grainy-sounding noise types, e.g., optical soundtrack noise of celluloid movie footage, or rain noise.

Harmonic Phase Estimation in Single-Channel Speech Enhancement Using Phase Decomposition and SNR Information

In conventional single-channel speech enhancement, typically the noisy spectral amplitude is modified while the noisy phase is used to reconstruct the enhanced signal. Several recent attempts have shown the effectiveness of utilizing an improved spectral phase for phase-aware speech enhancement and consequently its positive impact on the perceived speech quality. In this paper, we present a harmonic phase estimation method relying on fundamental frequency and signal-to-noise ratio (SNR) information estimated from noisy speech. The proposed method relies on SNR-based time-frequency smoothing of the unwrapped phase obtained from the decomposition of the noisy phase. To incorporate the uncertainty in the estimated phase due to unreliable voicing decision and SNR estimate, we propose a binary hypothesis test assuming speech-present and speech-absent classes representing high and low SNRs. The effectiveness of the proposed phase estimation method is evaluated for both phase-only enhancement of noisy speech and in combination with an amplitude-only enhancement scheme. We show that by enhancing the noisy phase both perceived speech quality as well as speech intelligibility are improved as predicted by the instrumental metrics and justified by subjective listening tests.

ALISA: An automatic lightly supervised speech segmentation and alignment tool

This paper describes the ALISA tool, which implements a lightly supervised method for sentence-level alignment of speech with imperfect transcripts. Its intended use is to enable the creation of new speech corpora from a multitude of resources in a language-independent fashion, thus avoiding the need to record or transcribe speech data. The method is designed so that it requires minimum user intervention and expert knowledge, and it is able to align data in languages which employ alphabetic scripts. It comprises a GMM-based voice activity detector and a highly constrained grapheme-based speech aligner. The method is evaluated objectively against a gold standard segmentation and transcription, as well as subjectively through building and testing speech synthesis systems from the retrieved data. Results show that on average, 70% of the original data is correctly aligned, with a word error rate of less than 0.5%. In one case, subjective listening tests show a statistically significant preference for voices built on the gold transcript, but this is small and in other tests, no statistically significant differences between the systems built from the fully supervised training data and the one which uses the proposed method are found.

Acoustic Source Localization and Deconvolution-Based Separation

This paper examines two fundamental issues in sound field analysis: acoustic sources localization and separation. Algorithms are developed to locate and separate acoustic signals on the basis of plane-wave decomposition. In the localization stage, directions of plane waves are determined using either minimum variance distortionless response (MVDR) method or multiple signal classification (MUSIC) method. For broadband scenarios, coherent and incoherent techniques are utilized in the localization procedure. In the separation stage, two approaches with overdetermined and underdetermined settings can be employed. In the overdetermined approach, Tikhonov regularization (TIKR) is utilized to recover the source signals. In the underdetermined approach, the steering matrix is augmented by including the directions that have been determined in the localization stage. Hence, the separation problem is formulated into a compressive sensing (CS) problem which can be effectively solved by using convex (CVX) optimization. Simulation and experiments are conducted for a 24-element circular array. Objective tests using perceptual evaluation of speech quality (PESQ) tests and subjective listening tests demonstrate that the proposed methods yield speech signals with well separated and improved quality, as compared to the mixed signals.

Annoyance of wind-turbine noise as a function of amplitude-modulation parameters

Amplitude modulation (AM) has been suggested as an important factor for the perceived annoyance of wind-turbine noise (WTN). Two AM types, typically referred to as “normal AM” and “other AM,” depending on the AM extent and frequency region, have been proposed to characterize WTN AM. The extent to which AM depth, frequency, and type affect WTN annoyance is a matter of debate. In most subjective studies, the temporal variations of WTN AM have not been considered. Here, a sinusoidally modulated WTN model accounting for temporal AM variations was used to generate realistic artificial stimuli in which the AM depth, frequency, and type, while determined from real on-site recordings, could be varied systematically. Subjective listening tests with such stimuli showed that a reduction in AM depth, quantified by the modulation depth spectrum, led to a significant decrease in annoyance. When the spectrotemporal characteristics of the original far-field stimuli were included in the model and the temporal AM variations were taken into account by varying the modulation index over time, neither AM frequency nor AM type were found to significantly affect annoyance. These findings suggest that the effect of AM parameters on WTN annoyance may depend on the intermittent nature of WTN AM.

Driver Filter Design for Software-Implemented Loudspeaker Crossovers

Abstract A hybrid method is presented for the integration of low-, mid-, and high-frequency driver filters in loud-speaker crossovers. The Pascal matrix is exploited to calculate denominators; the locations of minimum values in frequency magnitude responses are associated with the forms of numerators; the maximum values are used to compute gain factors. The forms of the resulting filters are based on the physical meanings of low-pass, band-pass, and high-pass filters, an intuitive idea which is easy to be understood. Moreover, each coefficient is believed to be simply calculated, an advantage which keeps the software-implemented crossover running smoothly even if crossover frequencies are being changed in real time. This characteristic allows users to efficiently adjust the bandwidths of the driver filters by subjective listening tests if objective measurements of loudspeaker parameters are unavailable. Instead of designing separate structures for a low-, mid-, and high-frequency driver filter, by using the proposed techniques we can implement one structure which merges three types of digital filters. Not only does the integration architecture operate with low computational cost, but its size is also compact. Design examples are included to illustrate the effectiveness of the presented methodology

Objective measures for quality assessment of noise-suppressed speech

Among all the existing objective measures, few are able to provide a clearly specific indication of speech distortion or noise reduction, which are the two key metrics to assess the performance of speech enhancement algorithms and evaluate the noise-suppressed speech quality. In this paper, new quantitative quality assessments are proposed to separately evaluate the capabilities of single channel speech enhancement algorithms in terms of maintaining the clean speech, noise reduction and overall performance. Based on these aspects, three evaluation results can be provided for any one test speech signal by analyzing the residual signal which is the difference between the clean speech and the processed speech. Several common speech enhancement algorithms are compared by these objective measures as well as subjective listening tests. High correlations between the scores of the objective measures and subjective ratings clearly show the effectiveness of the proposed evaluation methodologies on the different speech enhancement algorithms.

Effect of Vertical Microphone Layer Spacing for a 3D Microphone Array

Subjective listening tests were conducted to investigate how the spacing between main (lower) and height (upper) microphone layers in a 3D main microphone array affects perceived spatial impression and overall preference. Four different layer spacings of 0m, 0.5m, 1m, and 1.5m were compared for the sound sources of trumpet, acoustic guitar, percussion quartet, and string quartet using a nine-channel loudspeaker setup. It was generally found that there was no significant difference between any of the spaced layer configurations, whereas the 0m layer had slightly higher ratings than the more spaced layers in both spatial impression and preference. Acoustical properties of the original microphone channel signals as well as those of the reproduced signals, which were binaurally recorded, were analyzed in order to find possible physical causes for the perceived results. It is suggested that the perceived results were mainly associated with vertical interchannel crosstalk in the signals of each height layer and the magnitude and pattern of spectral change at the listener’s ear caused by each layer.

Audio bandwidth extension based on temporal smoothing cepstral coefficients

In this paper, we propose a wideband (WB) to super-wideband audio bandwidth extension (BWE) method based on temporal smoothing cepstral coefficients (TSCC). A temporal relationship of audio signals is included into feature extraction in the bandwidth extension frontend to make the temporal evolution of the extended spectra smoother. In the bandwidth extension scheme, a Gammatone auditory filter bank is used to decompose the audio signal, and the energy of each frequency band is long-term smoothed using minima controlled recursive averaging (MCRA) in order to suppress transient components. The resulting ‘steady-state’ spectrum is processed by frequency weighting, and the temporal smoothing cepstral coefficients are obtained by means of the power-law loudness function and cepstral normalization. The extracted temporal smoothing cepstral coefficients are fed into a Gaussian mixture model (GMM)-based Bayesian estimator to estimate the high-frequency (HF) spectral envelope, while the fine structure is restored by spectral translation. Evaluation results show that the temporal smoothing cepstral coefficients exploit the temporal relationship of audio signals and provide higher mutual information between the low- and high-frequency parameters, without increasing the dimension of input vectors in the frontend of bandwidth extension systems. In addition, the proposed bandwidth extension method is applied into the G.729.1 wideband codec and outperforms the Mel frequency cepstral coefficient (MFCC)-based method in terms of log spectral distortion (LSD), cosh measure, and differential log spectral distortion. Further, the proposed method improves the smoothness of the reconstructed spectrum over time and also gains a good performance in the subjective listening tests.

A uniform phase representation for the harmonic model in speech synthesis applications

Feature-based vocoders, e.g., STRAIGHT, offer a way to manipulate the perceived characteristics of the speech signal in speech transformation and synthesis. For the harmonic model, which provide excellent perceived quality, features for the amplitude parameters already exist (e.g., Line Spectral Frequencies (LSF), Mel-Frequency Cepstral Coefficients (MFCC)). However, because of the wrapping of the phase parameters, phase features are more difficult to design. To randomize the phase of the harmonic model during synthesis, a voicing feature is commonly used, which distinguishes voiced and unvoiced segments. However, voice production allows smooth transitions between voiced/unvoiced states which makes voicing segmentation sometimes tricky to estimate. In this article, two-phase features are suggested to represent the phase of the harmonic model in a uniform way, without voicing decision. The synthesis quality of the resulting vocoder has been evaluated, using subjective listening tests, in the context of resynthesis, pitch scaling, and Hidden Markov Model (HMM)-based synthesis. The experiments show that the suggested signal model is comparable to STRAIGHT or even better in some scenarios. They also reveal some limitations of the harmonic framework itself in the case of high fundamental frequencies.

Exemplar-Based Sparse Representation With Residual Compensation for Voice Conversion

We propose a nonparametric framework for voice conversion, that is, exemplar-based sparse representation with residual compensation. In this framework, a spectrogram is reconstructed as a weighted linear combination of speech segments, called exemplars, which span multiple consecutive frames. The linear combination weights are constrained to be sparse to avoid over-smoothing, and high-resolution spectra are employed in the exemplars directly without dimensionality reduction to maintain spectral details. In addition, a spectral compression factor and a residual compensation technique are included in the framework to enhance the conversion performances. We conducted experiments on the VOICES database to compare the proposed method with a large set of state-of-the-art baseline methods, including the maximum likelihood Gaussian mixture model (ML-GMM) with dynamic feature constraint and the partial least squares (PLS) regression based methods. The experimental results show that the objective spectral distortion of ML-GMM is reduced from 5.19 dB to 4.92 dB, and both the subjective mean opinion score and the speaker identification rate are increased from 2.49 and 73.50% to 3.15 and 79.50%, respectively, by the proposed method. The results also show the superiority of our method over PLS-based methods. In addition, the subjective listening tests indicate that the naturalness of the converted speech by our proposed method is comparable with that by the ML-GMM method with global variance constraint.

OP31 * THE SOUND OF SCIENCE: DEVELOPING THE SPECTRAL LIGHT ORCHESTRA

INTRODUCTION: Complete tumour removal during surgery is one of the primary factors for recurrence-free survival. The current process is unable to accurately identify tumour margins and data feedback takes too long for responsive action during surgery. Our recent study has shown the ability of Raman spectroscopy to distinguish between metastatic brain, GBM and normal brain tissue to sensitivities and specificities ranging from 85.71% to 100%. Pushing this result towards the clinic needs an assessment of the data feedback. In a surgery situation, audio feedback of tumour margins would provide a real time response allowing the surgeons visual focus to remain on the patient.Auditory perception of complex, structured information could have several advantages when compared to visual images including the capability of the human ear to detect patterns, recognise timbres and follow different strands at the same time. METHOD: Our method consists of two phases, to compute the relevant variables, we apply feature extraction to the data, extracting band-wise spectral shape descriptors such as centroid, skew and kurtosis. These are then used to modulate parameters of a synthesizer. RESULTS: The method was applied to a dataset of 311 samples, 136 from an uncontrolled and 176 from a controlled group. Based on subjective listening tests taken from around 20 participants, we were able to get manual classification accuracies of up to 83%, using a multi-stimulus technique. CONCLUSION: Raman spectroscopy is able to accurately distinguish between cancerous and non-cancerous tissue. The development of novel data extraction tools has enabled audio feedback of these molecular differences in real time allowing the surgeons visual focus to remain on the patient.

Temporal analysis, manipulation, and resynthesis of musical vibrato

Vibrato is an important music performance technique for both voice and various music instruments. In this paper, a signal processing framework for vibrato analysis, manipulation and resynthesis is presented. In the analysis part, music vibrato is treated as a generalized descriptor of music timbre and the signal magnitude and instantaneous frequency are implemented as temporal features. Specifically, the magnitude track shows the dynamic variations of audio loudness, and the frequency track shows the frequency deviations varying with time. In the manipulation part, several manipulation methods for the magnitude track and the frequency track are implemented. The tracking results are manipulated in both the time- and the frequency-domain. These manipulation methods are implemented as an interactive process to allow musicians to manually adjust the processing parameters. In the resynthesis part, the simulated vibrato audio is created using sinusoidal resynthesis process. The resynthesis part serves three purposes: to imitate human music performance, to migrate sonic features across music performances, and to serve as creative audio design tools, e.g., to create non-existing vibrato characteristics. The source audio from human music performance and the resynthesized audio are compared using subjective listening tests to validate our proposed framework.

Multi-channel audio recording based on superdirective beamforming for portable multimedia recording devices

In this paper, an audio recording method using a near-coincident microphone array is proposed for a multichannel audio system. A coincident or near-coincident microphone recording method attempts to capture spatial images in a multi-channel audio system depending on directional-level differences or directivities. Thus, high directivity is required for good spatial quality in the multichannel audio system, and it is usually obtained using a directional microphone. However, the currently available microphones such as cardioid, hyper-cardioid, and figure-8 directional microphones have lower directivities. To overcome this problem, the proposed method forms a superdirective sound beam in each steered direction using closely spaced omni-microphones. It is shown from subjective listening tests that the proposed method provides higher spatial quality with lower timbral distortions than other conventional recording methods for stereo and 5.1-channel audio systems.

Modeling individual HRTF tensor using high-order partial least squares

A tensor is used to describe head-related transfer functions (HRTFs) depending on frequencies, sound directions, and anthropometric parameters. It keeps the multi-dimensional structure of measured HRTFs. To construct a multi-linear HRTF personalization model, an individual core tensor is extracted from the original HRTFs using high-order singular value decomposition (HOSVD). The individual core tensor in lower-dimensional space acts as the output of the multi-linear model. Some key anthropometric parameters as the inputs of the model are selected by Laplacian scores and correlation analyses between all the measured parameters and the individual core tensor. Then, the multi-linear regression model is constructed by high-order partial least squares (HOPLS), aiming to seek a joint subspace approximation for both the selected parameters and the individual core tensor. The numbers of latent variables and loadings are used to control the complexity of the model and prevent overfitting feasibly. Compared with the partial least squares regression (PLSR) method, objective simulations demonstrate the better performance for predicting individual HRTFs especially for the sound directions ipsilateral to the concerned ear. The subjective listening tests show that the predicted individual HRTFs are approximate to the measured HRTFs for the sound localization.