Nonlinear Speech Research Articles

A Novel Method for Intelligibility Assessment of Nonlinearly Processed Speech in Spaces Characterized by Long Reverberation Times.

Objective assessment of speech intelligibility is a complex task that requires taking into account a number of factors such as different perception of each speech sub-bands by the human hearing sense or different physical properties of each frequency band of a speech signal. Currently, the state-of-the-art method used for assessing the quality of speech transmission is the speech transmission index (STI). It is a standardized way of objectively measuring the quality of, e.g., an acoustical adaptation of conference rooms or public address systems. The wide use of this measure and implementation of this method on numerous measurement devices make STI a popular choice when the speech-related quality of rooms has to be estimated. However, the STI measure has a significant drawback which excludes it from some particular use cases. For instance, if one would like to enhance speech intelligibility by employing a nonlinear digital processing algorithm, the STI method is not suitable to measure the impact of such an algorithm, as it requires that the measurement signal should not be altered in a nonlinear way. Consequently, if a nonlinear speech enhancing algorithm has to be tested, the STI—a standard way of estimating speech transmission cannot be used. In this work, we would like to propose a method based on the STI method but modified in such a way that it makes it possible to employ it for the estimation of the performance of the nonlinear speech intelligibility enhancement method. The proposed approach is based upon a broadband comparison of cumulated energy of the transmitted envelope modulation and the received modulation, so we called it broadband STI (bSTI). Its credibility with regard to signals altered by the environment or nonlinear speech changed by a DSP algorithm is checked by performing a comparative analysis of ten selected impulse responses for which a baseline value of STI was known.

English spoken stress recognition based on natural language processing and endpoint detection algorithm

Nowadays, there are more and more researches on the application of natural language processing technology in computer-aided language system, which can provide a good assistant role for foreign language learners. However, in the research of computer-aided language system, there are still some deficiencies in the recognition of English spoken stress nodes, which cannot be well recognized. Based on this, this paper proposes a method of English spoken accent recognition based on natural language processing and endpoint detection algorithm, which aims to promote the accuracy of accent recognition in the computer-aided language system and improve the performance of the computer-aided language system. In order to avoid the interference of background noise, this paper proposes a short-term time-frequency endpoint detection algorithm which can accurately judge the beginning and end of speech in complex environment. Then, on the basis of traditional speech feature extraction and fractal dimension theory, a nonlinear fractal dimension speech feature is extracted. Finally, RankNet is used to process the extracted features to realize the recognition of English spoken stress nodes. In the simulation analysis, the application effect of the short-term time-frequency endpoint detection algorithm proposed in this paper in the complex background noise and the effect of non-linear fractal dimension speech features on the recognition of English spoken stress nodes are verified. Finally, the performance and good application effect of the method designed in this paper are illustrated.

Complex Neural Spatial Filter: Enhancing Multi-Channel Target Speech Separation in Complex Domain

To date, mainstream target speech separation (TSS) approaches are formulated to estimate the complex ratio mask (cRM) of the target speech in time-frequency domain under supervised deep learning framework. However, the existing deep models for estimating cRM are designed in the way that the real and imaginary parts of the cRM are separately modeled using real-valued training data pairs. The research motivation of this study is to design a deep model that fully exploits the temporal-spectral-spatial information of multi-channel signals for estimating cRM directly and efficiently in complex domain. As a result, a novel TSS network is designed consisting of two modules, a complex neural spatial filter (cNSF) and an MVDR. Essentially, cNSF is a cRM estimation model and an MVDR module is cascaded to the cNSF module to reduce the nonlinear speech distortions introduced by neural network. Specifically, to fit the cRM target, all input features of cNSF are reformulated into complex-valued representations following the supervised learning paradigm. Then, to achieve good hierarchical feature abstraction, a complex deep neural network (cDNN) is delicately designed with U-Net structure. Experiments conducted on simulated multi-channel speech data demonstrate the proposed cNSF outperforms the baseline NSF by 12.1% scale-invariant signal-to-distortion ratio and 33.1% word error rate.

Дискурсивная личность персонажа

The discursive personality of the character is manifested in a set of fragments and text units that characterize the character’s speech style, expressed by a number of individual semantic and stylistic, communicative and pragmatic, cognitive, gender, socio-cultural and psychological characteristics. The discursive personality of a character is a complex structure and includes characteristics inherent in the character and the author. Speech units, expressed in the discursive personality of the character and the author, refer to artistic dialogue, graphic means, and non-linear speech.

A novel pipelined neural IIR adaptive filter for speech prediction

The paper presents a pipelined neural IIR filter (PNIIR) for nonlinear speech prediction. It inherits the usual pipelined two layers architecture: the nonlinear modular cascaded subsections and linear combiner subsection, and the nonlinear and linear weights of each module are updated using an real-time learning algorithms. The PNIIR filter units the good tracking performance of the neural IIR network and the low computation load of the pipelined architecture. The performance analysis and complexity analysis are illustrated in this paper. The experimental study for speech prediction is also carried out to testify the efficiency of the proposed nonlinear filter.

Effects of linear and nonlinear speech rate changes on speech intelligibility in stationary and fluctuating maskers

Algorithmic modifications to the durational structure of speech designed to avoid intervals of intense masking lead to increases in intelligibility, but the basis for such gains is not clear. The current study addressed the possibility that the reduced information load produced by speech rate slowing might explain some or all of the benefits of durational modifications. The study also investigated the influence of masker stationarity on the effectiveness of durational changes. Listeners identified keywords in sentences that had undergone linear and nonlinear speech rate changes resulting in overall temporal lengthening in the presence of stationary and fluctuating maskers. Relative to unmodified speech, a slower speech rate produced no intelligibility gains for the stationary masker, suggesting that a reduction in information rate does not underlie intelligibility benefits of durationally modified speech. However, both linear and nonlinear modifications led to substantial intelligibility increases in fluctuating noise. One possibility is that overall increases in speech duration provide no new phonetic information in stationary masking conditions, but that temporal fluctuations in the background increase the likelihood of glimpsing additional salient speech cues. Alternatively, listeners may have benefitted from an increase in the difference in speech rates between the target and background.

Fuzzy-based algorithm for Fongbe continuous speech segmentation

Text-independent speech segmentation is a challenging topic in computer-based speech recognition systems. This paper proposes a novel time-domain algorithm based on fuzzy knowledge for continuous speech segmentation task via a nonlinear speech analysis. Short-term energy, zero-crossing rate and the singularity exponents are the time-domain features that we have calculated in each point of speech signal in order to exploit relevant information for generating the significant segments. This is down for the phoneme or syllable identification and the transition fronts. Fuzzy logic technique helped us to fuzzify the calculated features into three complementary sets namely: low, medium, high and to perform a matching phase using a set of fuzzy rules. The outputs of our proposed algorithm are silence, phonemes, or syllables. Once evaluated, our algorithm produced the best performances with efficient results on Fongbe language (an African tonal language spoken especially in Benin, Togo and Nigeria).

Analysis of acoustic features for speech intelligibility prediction models analysis of acoustic features for speech intelligibility prediction models

We have developed a new model to predict speech intelligibility of synthetic sounds processed by nonlinear speech enhancement algorithms. The model involves two recent auditory models: the dynamic compressive gammachirp (dcGC) auditory filterbank and the speech envelope power spectrum model (sEPSM). The dcGC-sEPSM was compared with commonly used prediction models based on perceptual intelligibility scores of speech sounds enhanced by classic spectral subtraction and state-of-the-art Wiener filtering. As a result, the dcGC-sEPSM predicted the scores better than the coherence SII (CSII), the short-time objective intelligibility (STOI), and the original sEPSM using the gammatone filterbank. There was, however, still inconsistency between the prediction and data. In this work, we show the analysis of acoustic features used in the prediction models. The CSII calculates the magnitude-squared coherence between clean and processed spectra to derive a signal-to-distortion ratio. The STOI calculates the correlation coefficients between the short-time frame vectors of clean and degraded sound at the output of the one-third octave filterbank. The sEPSM calculates the signal-to-noise ratio in the envelope modulation domain at the output of the auditory filterbank. We summarize the methods and discuss desirable features that improve speech intelligibility predictions.

Complexity and synchronization in stochastic chaotic systems

We investigate the complexity of a hyperchaotic dynamical system perturbed by noise and various nonlinear speech and music signals. The complexity is measured by the weighted recurrence entropy of the hyperchaotic and stochastic systems. The synchronization phenomenon between two stochastic systems with complex coupling is also investigated. These criteria are tested on chaotic and perturbed systems by mean conditional recurrence and normalized synchronization error. Numerical results including surface plots, normalized synchronization errors, complexity variations etc show the effectiveness of the proposed analysis.

Nonlinear speech enhancement by virtual increase of channels and maximum SNR beamformer

In this paper, we propose a new microphone array signal processing technique, which increases the number of microphones virtually by generating extra signal channels from real microphone signals. Microphone array signal processing methods such as speech enhancement are effective for improving the quality of various speech applications such as speech recognition and voice communication systems. However, the performance of speech enhancement and other signal processing methods depends on the number of microphones. Thus, special equipment such as a multichannel A/D converter or a microphone array is needed to achieve high processing performance. Therefore, our aim was to establish a technique for improving the performance of array signal processing with a small number of microphones and, in particular, to increase the number of channels virtually by synthesizing virtual microphone signals, or extra signal channels, from two channels of microphone signals. Each virtual microphone signal is generated by interpolating a short-time Fourier transform (STFT) representation of the microphone signals. The phase and amplitude of the signal are interpolated individually. The phase is linearly interpolated on the basis of a sound propagation model, and the amplitude is nonlinearly interpolated on the basis of β divergence. We also performed speech enhancement experiments using a maximum signal-to-noise ratio (SNR) beamformer equipped with virtual microphones and evaluated the improvement in performance upon introducing virtual microphones.

Nonlinear Speech Analysis and Modeling for Malayalam Vowel Recognition

This paper presents a novel method for the recognition of Malayalam vowel phonemes using nonlinear speech parameters such as maximal Lyapunov exponent and Phase Space Anti-diagonal Point Distribution. Reconstructed phase space is used as a base for extracting these features. The results show that the proposed nonlinear features have significant discriminatory power. The combined feature vectors of nonlinear feature and MFCC features yield increased recognition accuracy for five Malayalam vowel phonemes.

Feature selection for automatic analysis of emotional response based on nonlinear speech modeling suitable for diagnosis of Alzheimer׳s disease

Alzheimer׳s disease (AD) is the most common type of dementia among the elderly. This work is part of a larger study that aims to identify novel technologies and biomarkers or features for the early detection of AD and its degree of severity. The diagnosis is made by analyzing several biomarkers and conducting a variety of tests (although only a post-mortem examination of the patients’ brain tissue is considered to provide definitive confirmation). Non-invasive intelligent diagnosis techniques would be a very valuable diagnostic aid. This paper concerns the Automatic Analysis of Emotional Response (AAER) in spontaneous speech based on classical and new emotional speech features: Emotional Temperature (ET) and fractal dimension (FD). This is a pre-clinical study aiming to validate tests and biomarkers for future diagnostic use. The method has the great advantage of being non-invasive, low cost, and without any side effects. The AAER shows very promising results for the definition of features useful in the early diagnosis of AD.

The effect of longer range connections on neuronal network dynamics

The effect of longer range connections on neuronal network dynamics

Nonlinear speech coding model based on genetic programming

An improved genetic programming is proposed in this paper to construct the nonlinear models of speech signals, and the speech coding is further accomplished. After the preprocessing of the speech signals, the improved GP is used to construct the corresponding model of each speech frame. Then by analyzing these models, a normalized model that has generalization ability is obtained. And finally the process of speech coding is accomplished by the optimizing the parameters of the normalized model using an optimization algorithm. Experiments demonstrate that the feasibility of the improved GP in the modeling of speech signals, and show the superiority of the proposed method in speech coding based on the comparisons with the linear predictive coding.

Nonlinear spectro-temporal features based on a cochlear model for automatic speech recognition in a noisy situation

A nonlinear speech feature extraction algorithm was developed by modeling human cochlear functions, and demonstrated as a noise-robust front-end for speech recognition systems. The algorithm was based on a model of the Organ of Corti in the human cochlea with such features as such as basilar membrane (BM), outer hair cells (OHCs), and inner hair cells (IHCs). Frequency-dependent nonlinear compression and amplification of OHCs were modeled by lateral inhibition to enhance spectral contrasts. In particular, the compression coefficients had frequency dependency based on the psychoacoustic evidence. Spectral subtraction and temporal adaptation were applied in the time-frame domain. With long-term and short-term adaptation characteristics, these factors remove stationary or slowly varying components and amplify the temporal changes such as onset or offset. The proposed features were evaluated with a noisy speech database and showed better performance than the baseline methods such as mel-frequency cepstral coefficients (MFCCs) and RASTA-PLP in unknown noisy conditions.

Speaker Identification Using a Nonlinear Speech Model and ANN

This paper introduces a nonlinear speech model for improved speaker identification. We modelled the speaker identity using Reconstructed Phase Space (RPS) of the speech signal and the Phase Space Point Distribution (PSPD) parameters. The PSPD parameters are extracted from five vowels uttered by different speakers. The speaker identification experiments are conducted based on the PSPD parameters using Feed Forward Multi Layer Perceptron (FFMLP). Overall performance of the Mel-frequency cepstral baseline system is compared with the proposed composite classifier system using both cepstral and PSPD features across ten different speakers. The experimental results indicate that the accuracy of the phase space approach by itself is below that of MFCC features and further it shows that the proposed approach in which PSPD features when used with MFCC, pitch and first formant frequency offers reasonable improvement in speaker identification accuracy of the system.

The Speech Continuum Reconsidered: A Case Study of a Speaker of Indian English

This study highlights the dynamics of the speech continuum by analyzing the sound system of a speaker of Indian English (IndE). Utterances were collected via the reading of commonly spoken words and a semi-structured interview. The findings indicate that the speaker’s sound features are much fewer than those described in earlier studies on IndE accents. Moreover, his pronunciation patterns display a mixture of the basilectal, meoslectal and acrolectal forms of Indian English, reflecting the characteristics of the non-linear speech continuum. This study concludes by discussing the speaker’s sound patterns with relation to his sociolinguistic background.

Nonlinear enhancement of noisy speech, using continuous attractor dynamics formed in recurrent neural networks

Here, formation of continuous attractor dynamics in a nonlinear recurrent neural network is used to achieve a nonlinear speech denoising method, in order to implement robust phoneme recognition and information retrieval. Formation of attractor dynamics in recurrent neural network is first carried out by training the clean speech subspace as the continuous attractor. Then, it is used to recognize noisy speech with both stationary and nonstationary noise. In this work, the efficiency of a nonlinear feedforward network is compared to the same one with a recurrent connection in its hidden layer. The structure and training of this recurrent connection, is designed in such a way that the network learns to denoise the signal step by step, using properties of attractors it has formed, along with phone recognition. Using these connections, the recognition accuracy is improved 21% for the stationary signal and 14% for the nonstationary one with 0db SNR, in respect to a reference model which is a feedforward neural network.

Non-Linear and Non-Conventional Speech Processing: Alternative Techniques

This special issue aims to cover some problems related to non-linear and non-conventional speech processing. The origin of this volume is in the ISCA Tutorial and Research Workshop on Non-Linear Speech Processing, NOLISP’09, held at the Universitat de Vic (Catalonia, Spain) on June 25–27, 2009. The series of NOLISP workshops started in 2003 has become a biannual event whose aim is to discuss alternative techniques for speech processing that, in a sense, do not fit into mainstream approaches. A selected choice of papers based on the presentations delivered at NOLISP’09 has given rise to this issue of Cognitive Computation. The papers hereinafter deal with the following topics: What would happen if we were able to overcome the limitations of audiovisual human systems? The answer could be found in the paper by V. Espinosa-Duro, M. Faundez-Zanuy, and J. Mekyska. A growing recurrent self-organizing model for phoneme recognition is presented by Ch. Jlassi, N. Arous, and N. Ellouze. An algorithm for voiced/unvoiced decision and pitch estimation from speech signals based on classification of peaks using the auto-correlation of the speech multi-scale product is presented by M. A. Ben Massoud, A. Bouzid, and N. Elloze. A feature reduction system based on the Discriminative Common Vector is proposed by C.M. Travieso, M. del Pozo, M.A. Ferrer, and J. B. Alonso. A new approach to the diagnosis of Ataxia SCA-2 by application of independent component analysis to a set of data obtained by electro-oculography is presented by R. V. Garcia, F. Rojas, C. G. Puntonet, B. San Roman, L. Velazquez, and R. Rodriguez. A formant tracking technique based on Fourier ridges detection aiming at improving the performance of these algorithms is presented by I. Jemaa, K. Ouni and Y. Laprie A new architecture for vocabulary independent keyword detection for cognitive virtual agents (SEMAINE system) is proposed by M. Wollmer, F. Eyber, A. Graves, B. Schuller, and G. Rigoll. Their word spotting model is composed of a dynamic-Bayesian network and a bidirectional long–short-term memory recurrent neural network. An insight into non-linear transformations for improving the performance of an entropy-based voice activity detector is made in the paper by two guest editors (J. S. and V.Z.). By no means has the range of topics in this Special Issue covered all directions in non-linear speech processing. We would rather consider it a sample presenting several areas in progress. In any case, we hope that they will be of interest for the readers of Cognitive Computation. Our thanks are due to the Editor-in-Chief, Dr. Amir Hussain, for permanent support in preparation of this issue. Of course, this issue would have been impossible without the contributions of all authors. On the other hand, anonymous referees have also done a great job by carefully J. Sole-Casals (&) V. Zaiats Department of Digital Technologies and Information, Escola Politecnica Superior, Universitat de Vic, c/. de la Laura, 13, 08500 Vic (Barcelona), Spain e-mail: jordi.sole@uvic.cat

Nonnegative Matrix Factorization Applied to Nonlinear Speech and Image Cryptosystems

Nonnegative matrix factorization (NMF) is widely used in signal separation and image compression. Motivated by its successful applications, we propose a new cryptosystem based on NMF, where the nonlinear mixing (NLM) model with a strong noise is introduced for encryption and NMF is used for decryption. The security of the cryptosystem relies on following two facts: 1) the constructed multivariable nonlinear function is not invertible; 2) the process of NMF is unilateral, if the inverse matrix of the constructed linear mixing matrix is not nonnegative. Comparing with Lin's method (2006) that is a theoretical scheme using one-time padding in the cryptosystem, our cipher can be used repeatedly for the practical request, i.e., multitme padding is used in our cryptosystem. Also, there is no restriction on statistical characteristics of the ciphers and the plaintexts. Thus, more signals can be processed (successfully encrypted and decrypted), no matter they are correlative, sparse, or Gaussian. Furthermore, instead of the number of zero-crossing-based method that is often unstable in encryption and decryption, an improved method based on the kurtosis of the signals is introduced to solve permutation ambiguities in waveform reconstruction. Simulations are given to illustrate security and availability of our cryptosystem.