Received Speech Signal Research Articles

Joint enhancement and classification constraints for noisy speech emotion recognition

In the natural environment, the received speech signal is often interfered by noise, which reduces the performance of speech emotion recognition (SER) system. To this end, a noisy SER method based on joint constraints, including enhancement constraint and arousal-valence classification constraint (EC-AVCC), is proposed. This method introduces speech enhancement features and constraints into emotion recognition for the first time, which helps to improve the performance of noisy SER. It does not require denoising pre-processing for noisy speech, thus saving testing time. Specifically, it uses multi-domain statistical features (MDSF) composed of emotional and enhanced features as the input of the SER model based on convolution neural network (CNN) and long short-term memory-attention (ALSTM). In addition, the model is constrained jointly by speech enhancement and arousal-valence classification to obtain the robust and discriminative deep emotion features. Furthermore, in the auxiliary speech enhancement task, a joint loss function that simultaneously constrains the error of the ideal ratio mask and the error of the corresponding MDSF is introduced to obtain more robust features. The experimental results on the CASIA Chinese emotional dataset and the EMO-DB German emotional database show that compared with the CNN baseline, the proposed method improves the accuracy of SER in white noise and babble noise by 4.7%-9.9%.

Multiple time-instances features based approach for reference-free speech quality measurement

This paper investigates the problem of measuring speech quality of received speech signal without employing the original speech signal. The problem of deterioration of the speech quality arises due to noise present in the surroundings. To this line, we propose a multiple time-instances (MTI) features-based approach for reference-free speech quality measurement model. A voice activity detector (VAD) is exploited primarily for calculating the number of active speech chunks of a speech signal. For these chunks and their successive combinations called here batches, multi-resolution auditory model (MRAM), mel-frequency cepstral coefficients (MFCC) and line spectral frequencies (LSF) features are extracted and called as MTI features. It is hypothesized that the MTI features are capable in capturing the distortions caused by time-localized effects of short-time transients, impulsive noise, and its differences from the plosive sounds. The MTI metric estimates (MTI-ME) are calculated corresponding to these MTI features employing the Gaussian mixture model (GMM) probabilistic technique. The overall objective speech quality of a speech signal is then determined as a linear combination of optimally weighted MTI-ME corresponding to distinct active speech chunks and their successive combinations, that is, batches of that speech signal. Minimum mean square error criterion or Pearson’s correlation maximization criterion is employed for computing optimal weights. In addition, a deep neural network (DNN)-based speech quality model is also developed for calculating a single objective speech quality while considering all active speech chunks together. Pearson’s correlation coefficient and weighted average correlation are exploited for evaluating the performance. Results demonstrate that the proposed model achieves promising improvement over the standard speech quality model (P.563) and improves correlation values by around 37%.

Speech Enhancement Framework with Noise Suppression Using Block Principal Component Analysis

With the advancement in voice-communication-based human–machine interface technology in smart home devices, the ability to decompose the received speech signal into a signal of interest and an interference component has emerged as a key requirement for their successful operation. These devices perform their tasks in real time based on the received commands, and their effectiveness is limited when there is a lot of ambient noise in the area in which they operate. Most real-time speech enhancement algorithms do not perform adequately well in the presence of high amounts of noise (i.e., low input-signal-to-noise ratio). In this manuscript, we propose a speech enhancement framework to help these algorithms in situations when the noise level in the received signal is high. The proposed framework performs noise suppression in the frequency domain by generating an approximation of the noisy signals’ short-time Fourier transform, which is then used by the speech enhancement algorithms to recover the underlying clean signal. This approximation is performed by using the proposed block principal component analysis (Block-PCA) algorithm. To illustrate efficacy of the proposed framework, we present a detailed performance evaluation under different noise levels and noise types, highlighting the effectiveness of the proposed framework. Moreover, the proposed method can be used in conjunction with any speech enhancement algorithm to improve its performance under moderate to high noise scenarios.

An application layer forward error correction method of wireless network broadcasting communication for smart video surveillance system

Aiming at the wireless channel in which the state of video surveillance communication network changes dramatically, to ensure the video impact under a restricted channel bandwidth condition, an application layer forward error correction system for wireless video communication is proposed. This scheme is an improved forward error correction algorithm. The most appropriate redundancy amount is implemented for each FEC block by constantly altering the redundancy in FEC blocks. Firstly, the system describes and implements the optimum video transmission. Then, the RCPC code and Raptor code are used as forward error correction codes in physical layer and application layer respectively, in this way, the optimal system solution can be acquired. In the forward error correction technology, a feedback mechanism is then developed between the receiver and the sender, which will acknowledge that the receiver can extract the packet loss rate from the received speech signal in the current network state in real time, and then the packet loss rate will be transferred back to the sender by the feedback mechanism. The sender can modify the forward packet loss rate according to the packet loss rate. The error-correcting schema can help the next speech signal have the most appropriate forward error-correcting redundancy, reducing bandwidth waste, significantly improving the adaptability and efficiency of forward error-correcting technology in the wireless network. The simulation results show that the average video quality of the proposed scheme is 0.5–4 dB higher than that of the equal error protection scheme and the extended window unequal error protection scheme.

Improving the quality of speech in the conditions of noise and interference

The aim of the work is to present a method of intelligent modification of the speech signal with speech features expressed in noise, based on the Lombard effect. The recordings utilized sets of words and sentences as well as disturbing signals, i.e., pink noise and the so-called babble speech. Noise signal, calibrated to various levels at the speaker's ears, was played over two loudspeakers located 2 m away from the speaker. In addition, the recording session included utterances in quiet, which constitute a reference to the received speech signal analysis with the Lombard effect. As a part of the analysis, the following parameters were examined with regard to prosody: fundamental frequency F0, formant frequencies of F1 and F2, duration of the utterance, sound intensity, etc., taking into account individual sentences, words, and vowels. The PRAAT program was used to process and analyze speech signals. Next, a method for modifying speech with the features of speech spoken in noise was proposed. Subsequent analyzes have shown that noisy speech modified by the Lombard effect features is characterized by higher values of the PESQ (perceptual evaluation of speech quality) speech quality indicator compared to noisy speech without the features incorporated.The aim of the work is to present a method of intelligent modification of the speech signal with speech features expressed in noise, based on the Lombard effect. The recordings utilized sets of words and sentences as well as disturbing signals, i.e., pink noise and the so-called babble speech. Noise signal, calibrated to various levels at the speaker's ears, was played over two loudspeakers located 2 m away from the speaker. In addition, the recording session included utterances in quiet, which constitute a reference to the received speech signal analysis with the Lombard effect. As a part of the analysis, the following parameters were examined with regard to prosody: fundamental frequency F0, formant frequencies of F1 and F2, duration of the utterance, sound intensity, etc., taking into account individual sentences, words, and vowels. The PRAAT program was used to process and analyze speech signals. Next, a method for modifying speech with the features of speech spoken in noise was proposed. Subsequent a...

Blind digital speech watermarking using filter bank multicarrier modulation for 5G and IoT driven networks

Today’s world is a domain of Social Networks with millions of people sharing valuable information over the Internet. In the future wireless networks like 5G, it is estimated that 20 billion devices will be connected with the internet (IoT). Therefore, there are certain issues related to the privacy and security of a user while sharing information through a social network especially when the shared information carries multimedia content like images, video, audio/speech. The security issues such as integrity and data authentication can be fulfilled using various Digital Watermarking Techniques. In this paper a new technique of Speech Watermarking (Least-Significant-Bit substitution) using Filter Bank Multicarrier Modulation (FBMC) Technique for 5G networks is proposed and implemented. The technique has been implemented in MATLAB on a test recorded speech signal against various signal processing modifications (FBMC and Additive White Gaussian Noise). Experimental results reveal that the proposed technique in spite of various attacks allows to retrieve embedded watermark efficiently with less Bit Error Rate and better audio quality of the received speech signal.

Integrated different forms of business communication by using internet telephony

The paper provides an overview of development and use of Internet technologies in improving business by integrating different types of communication. The advantages of using the standard open IP - Internet Protocol in establishing integrated communication are presented as some of the potential possibilities of upgrading the existing conventional telephone network. The paper presents a concrete implementation of internet telephony in business premises in order to reduce operating costs, as well as the analysis of the possibilities of applying new multimedia applications and services by using a single integrated network. Comparison of performance of the original, sent speech signal and the received speech signal, after passing through the IP network, for different values of packet loss rate is analyzed.

Selective Frequency Enhancement of Speech Signal for Intelligibility Improvement in Presence of Near-end Noise

Abstract In mobile phones, perceived speech signal deteriorates significantly in the presence of background noise as noise reaches directly at the listener's ears. There is an inherent need to increase the intelligibility of the received speech signal in noisy environments by incorporating speech enhancement algorithm at the receiverend. This paper focus on the enhancement of selective speech samples for the intelligibility improvement, when the near-end noise dominates. Audible speech samples are selected by consideringthreshold of hearing and auditory masking properties of the human ear. Intelligibility of enhanced speech signalis measured using Speech Intelligibility Index. Experimental results show the intelligibility improvement of the speech signal in lower SNR with the proposed approach to the unprocessed far-end speech signal. This approach is efficient in overcoming the degradation of speech signals in very noisy environments.

Efficient speaker identification from speech transmitted over Bluetooth networks

This paper studies the process of speaker identification over Bluetooth networks. Bluetooth channel degradations are considered prior to the speaker identification process. The work in this paper employs Mel-frequency cepstral coefficients for feature extraction. Features are extracted from different transforms of the received speech signals such as the discrete cosine transform (DCT), signal plus DCT, discrete sine transform (DST), signal plus DST, discrete wavelet transform (DWT), and signal plus DWT. A neural network classifier is used in the experiments, while the training phase uses clean speech signals and the testing phase uses degraded signals due to communication over the Bluetooth channel. A comparison is carried out between the different methods of feature extraction showing that the DCT achieves the highest recognition rates.

Some effects of room acoustics and background noise on assistive listening systems and devices

Approximately 10–14% of the general population (United States and Northern Europe) suffer from a noticeable degree of hearing loss and would benefit from some form of hearing assistance or deaf aid. However, many hearing aids do not provide a suitable level of intelligibility when used in large reverberant or noisy spaces and rooms. The paper investigates the acoustic and speech intelligibility requirements for Assistive Listening Systems that may be used separately or in conjunction with a hearing aid to improve the potential intelligibility of the received speech signal. A number of microphone pick-up scenarios have been investigated and are reported in terms of their potential intelligibility and sound quality performance. The results of testing carried out in a number of rooms and venues are presented, mainly in terms of the resultant Speech Transmission Index (STI). The paper concludes by providing a number of recommendations and guidelines for successful microphone placement and introduces a novel technique for establishing useful talker sound radiation and hence microphone target aiming.

Accommodation Can Lead to Innovated Variation

UC Berkeley Phonology Lab Annual Report (2014) Accommodation can lead to innovated variation Jevon Heath University of California, Berkeley 1. Introduction The initiation of sound change requires the emergence of innovated variants. One possible source of innovated variants is non-faithful 1 transference of linguistic features during accommodation to a received speech signal. If the subphonemic cues associated with a phonological contrast are realized differently in an accommodating speaker's speech than in both the speaker's previous speech and in the signal to which they are accommodating, the resulting difference in phonetic realization may constitute a new variation. In this paper I argue that this sort of difference logically must happen in accommodation, at least for some speakers along some phonetic dimensions. I then report the results of a study demonstrating that such phonetic divergence in accommodation does happen in a laboratory environment, indicating that accommodation is a potential source of new phonetic variants. I conclude by situating these findings within extant work on sound change. To the extent that populations of individuals evince varying degrees of non- faithful transference of these cues, this study provides evidence for a potential approach to the actuation problem (Weinreich et al. 1968). 2. Background 2.1 Defining accommodation Accommodation is the phenomenon in which a person talks differently in response to external phonetic exposure. An often-reported example is the propensity of people calling their parents to revert to their speech. While talking differently is most commonly understood as talking more similarly to the received phonetic material (convergence), talking less similarly (divergence) has also been attested (cf. Giles 1971, Bourhis & Giles 1977, Babel 2010). However, divergence has only been evinced as moderated or intentional alteration of one's speech (cf. Bourhis & Giles 1977, Babel 2010), whereas convergence occurs both consciously and unconsciously (Bourhis & Giles 1977; Goldinger 1998). As such, convergence has commonly been considered default accommodative linguistic behavior. By non-faithful transference I mean a lack of phonetic fidelity between the signal perceived and the signal produced. I do not mean to invoke the idea of phonological faithfulness.

Speech Enhancement Algorithm to Reduce the Effect of Background Noise in Mobile Phones

In mobile phones, perceived quality of speech signal deteriorates significantly in the presence of background noise since near-end/surrounding noise also arrives at the near-end listener’s ears. The quality of the received signal varies widely depending upon signal strength and unavoidable background noise in the user environment. There is a need to improve the quality of received speech signal in noisy conditions by developing the speech enhancement algorithms. This paper focuses on the impact of the various background noises on signal degradation and mechanisms to mitigate the noise impact for improved speech signal perception. Gain adjustment process with simple time domain and frequency domain approach using psychoacoustic has been adapted to improve the quality and intelligibility of the speech signal in the noisy environments by automatically enhancing the speech signals when the noise dominates. Time domain approach is less complex and frequency domain using psychoacoustic is more efficient in overcoming the degradation of speech signals in presence of near-end or background noise.

SYSTEM AND METHOD OF USING ACOUSTIC MODELS FOR AUTOMATIC SPEECH RECOGNITION WHICH DISTINGUISH PRE− AND POST−VOCALIC CONSONANTS

Disclosed are systems, methods and computer readable media for training acoustic models for an automatic speech recognition systems (ASR) system. The method includes receiving a speech signal, defining at least one syllable boundary position in the received speech signal, based on the at least one syllable boundary position, generating for each consonant in a consonant phoneme inventory a pre-vocalic position label and a post-vocalic position label to expand the consonant phoneme inventory, reformulating a lexicon to reflect an expanded consonant phoneme inventory, and training a language model for an automated speech recognition (ASR) system based on the reformulated lexicon.

Prediction of the influence of reverberation on binaural speech intelligibility in noise and in quiet

Reverberation usually degrades speech intelligibility for spatially separated speech and noise sources since spatial unmasking is reduced and late reflections decrease the fidelity of the received speech signal. The latter effect could not satisfactorily be predicted by a recently presented binaural speech intelligibility model [Beutelmann et al. (2010). J. Acoust. Soc. Am. 127, 2479-2497]. This study therefore evaluated three extensions of the model to improve its predictions: (1) an extension of the speech intelligibility index based on modulation transfer functions, (2) a correction factor based on the room acoustical quantity "definition," and (3) a separation of the speech signal into useful and detrimental parts. The predictions were compared to results of two experiments in which speech reception thresholds were measured in a reverberant room in quiet and in the presence of a noise source for listeners with normal hearing. All extensions yielded better predictions than the original model when the influence of reverberation was strong, while predictions were similar for conditions with less reverberation. Although model (3) differed substantially in the assumed interaction of binaural processing and early reflections, its predictions were very similar to model (2) that achieved the best fit to the data.

Noise reduction in subbanded speech signals

The presence of speech in a filtered speech signal is detected for the purpose of suspending noise level calculations during periods of speech. A received speech signal is split into a plurality of subband signals. A subband variable gain is determined for each subband based on an estimation of the noise level in the received voice signal and on an envelope of the received signal in each subband. Each subband signal is multiplied by the subband variable gain for that subband. The subband signals are combined to produce an output voice signal.

Speech Quality Measurement Methods with Applying PLC Algorithms on Real-time Transmission Control Scheme for VoIP Service

According to the discovery of available and suitable audiovisual services over IP, researches on the Quality of Service (QoS) for various multimedia application services are required. This paper presents evaluation of speech quality applying improved Packet Loss Concealment (PLC) algorithms with an end-to-end bidirectional transmission control scheme. We propose real-time transmission control scheme to perform better quality monitoring of VoIP service. In order to evaluate the newly proposed RTCP-XR scheme for VoIP services, we measure in real-time the quality degree of the received speech signal while applying our improved Packet Loss Concealment (PLC) algorithms in G.711 and G.729A codecs using an implemented Session Initiation Protocol (SIP)- based SoftPhone and a 1-port gateway systems. Overall, for the evaluation with scientific exactitude of speech quality applied our PLC algorithms we examine with made phone-calls through the heterogeneous network systems. The experimental results confirm the applied PLC algorithms for improvement of speech quality over IP in real environments with the QoSfactor transmission control mechanism.

Contextual effects on the continuancy of /■/

This research examined the segmental-contextual effects on the continuancy of the voiced dental /■/ in American-English and whether certain acoustic attributes were preserved despite possible modification. Word-initial /■/ cases, extracted from continuous speech of 146 speakers from the TIMIT database, were frequently (averaging 65%) stoplike when it was in utterance-initial position or when its preceding phoneme was voiceless and/or [-continuant]. This stoplike modification occurred less frequently (averaging 39%) when /■/ was preceded by a voiced fricative and rarely (averaging 13%) when preceded by a vowel or liquid consonant. A comparison of stoplike /■/ and /d/ cases under similar contexts showed that the burst peak location, burst spectrum shape, and F2 at vowel onset averages were all statistically different between the two groups. In addition, the acoustic data suggested that the dental place of articulation was preserved for the modified /■/. Preliminary automatic classification experiments involving salient acoustic attributes seemed to indicate that F2 at vowel onset may be a reliable cue for the dental place of articulation in /■/. [Work supported by NIH DC02978 and T32DC00038.]

A Speech Packet Loss Concealment Method Using Linear Prediction

We proposed and evaluated a speech packet loss concealment method which predicts lost segments from speech included in packets either before, or both before and after the lost packet. The lost segments are predicted recursively by using linear prediction both in the forward direction from the packet preceding the loss, and in the backward direction from the packet succeeding the lost segment. Predicted samples in each direction are smoothed by averaging using linear weights to obtain the final interpolated signal. The adjacent segments are also smoothed extensively to significantly reduce the speech quality discontinuity between the interpolated signal and the received speech signal. Subjective quality comparisons between the proposed method and the the packet loss concealment algorithm described in the ITU standard G.711 Appendix I showed similar scores up to about 10% packet loss. However, the proposed method showed higher scores above this loss rate, with Mean Opinion Score rating exceeding 2.4, even at an extremely high packet loss rate of 30%. Packet loss concealment of speech degraded with G.729 coding, and babble noise mixed speech showed similar trends, with the proposed method showing higher qualities at high loss rates. We plan to further improve the performance by using adaptive LPC prediction order depending on the estimated pitch, and adaptive LPC bandwidth expansion depending on the consecutive number of repetitive prediction, among many other improvements. We also plan to investigate complexity reduction using gradient LPC coefficient updates, and processing delay reduction using adaptive forward/bidirectional prediction modes depending on the measured packet loss ratio.

Multiple wireless microphone speakerphone system and method

A multiple wireless microphone speakerphone system includes one or more wireless microphones. The wireless microphones accept speech and transmit the speech to receivers, one receiver corresponding to each wireless microphone. Audio processing and acoustic echo cancellation (AEC) are performed on the received speech signals. A signal handling unit generates an audio signal after echo cancellation. The generated audio signal is transmitted to a far end to be played through, for example, a speaker or an ear piece.

Speech Quality Enhancement Using Wavelet Reconstruction Filters

The present paper describes a quality enhancement of band-limited speech signals. In regular telephone communication, the quality of the received speech signal is degraded by band limitation. We propose an effective but simple scheme for obtaining narrowband speech signals in which the frequency components are estimated from band limited signals. The proposed method utilizes aliasing components generated by wavelet reconstruction filters in the inverse discrete wavelet transform. The results of enhancement have been verified by applying this method to speech samples via telephone lines to obtain a noticeable improvement in speech quality.