Speech Sources Research Articles

Clustering of spatial cues by semantic segmentation for anechoic binaural source separation

The recent introduction of neural networks to speech separation has dramatically boosted the separation performance. This paper presents a novel psychoacoustic approach for speech source separation in anechoic conditions, using semantic segmentation of the interaural spectrograms of the audio mixtures. We have trained two separate U-Nets (a specialized neural network for semantic segmentation) on the interaural level difference (ILD) spectrogram, and the interaural phase difference (IPD) spectrogram of a single source. After training, these U-Nets are used to predict the class of each time frequency (TF) unit of the interaural spectrogram of the audio mixture. The ILD and IPD soft masks obtained from these U-Nets are combined by a novel scheme which utilizes the strength of the interaural cues in different frequency bands. The results show improved separation over two state of the art machine learning source separation systems utilizing the same interaural cues. There is average improvement of 7.32 dB in signal to distortion ratio (SDR) and 0.3 points improvement in short term objective intelligibility (STOI) over degenerate un-mixing estimation technique (DUET) algorithm and 2.51 dB improvement in SDR with comparable intelligibility over model-based expectation–maximization source separation and localization (MESSL) algorithm.

Assessing the effect of visual servoing on the performance of linear microphone arrays in moving human-robot interaction scenarios

Social robotics is becoming a reality and voice-based human-robot interaction is essential for a successful human-robot collaborative symbiosis. The main objective of this paper is to assess the effect of visual servoing in the performance of a linear microphone array regarding distant ASR in a mobile, dynamic and non-stationary robotic testbed that can be representative of real HRI scenarios. Visual servoing and image target tracking are different tasks, and this paper focuses on an effect that is rarely addressed in the literature: the dependence of the beamforming directivity on look direction. The datasets required to carry out the study reported here do not exist and had to be generated. A state-of-the-art mobile robotic testbed had to be set up with target speech and noise sources. A linear microphone array was chosen as a case of study and its response was measured. Standard beamforming methods were evaluated with respect to visual servoing: delay-and-sum combined with image tracking; weighted delay-and-sum; and, MVDR also combined with image tracking. The results presented here show that the performance of beamforming methods is dramatically degraded in moving and non-stationary conditions. In this context, visual servoing in HRI can significantly improve the performance of a linear microphone array regarding ASR accuracy. The average reduction in WER achieved when the robot head was steered toward the target speech source was as high as 28.2%. Finally, it is worth highlighting that the methodology adopted here is applicable to any microphone array, linear or not.

Türkçe Öğretiminde “Hazırlıksız Konuşma” Sorunu

Konuşma, dinlemeyle birlikte geliştirilen bir öğrenme alanıdır. Çocuklar önce dinlemeyi sonra konuşmayı öğrenirler. Konuşma okuma ve yazma becerilerinin alt yapısını oluşturmaktadır. Bu nedenle çoğu ülkede konuşma öğretimine ayrı bir önem ağırlık verilmektedir. Bu amaçla dil öğretimi programları özenle hazırlanmakta, konuşmanın bilimsel süreç ve temelleri dikkate alınmaktadır. Günümüz dil öğretim yaklaşım ve teorilerine göre konuşma üç aşamada gerçekleşmektedir. Bunlar zihinsel hazırlık, zihinde düzenleme ve sözlere aktarmadır. Öğrencilerin konuşma becerilerini geliştirmek için bu aşamalara ağırlık verilmesi gerekmektedir. Oysa ülkemizdeki bazı kaynaklarda ve Türkçe öğretim programlarında hazırlıksız konuşmadan bahsedilmektedir. Bu çalışmada konuşma konusunda çeşitli kaynaklar ile ilk ve ortaokul Türkçe öğretim programlarındaki konuşma kazanımları incelenmiştir. İnceleme sonucunda 2017,2018 ve 2019 Türkçe Öğretim Programlarında verilen konuşma kazanımlarının dil öğretim yaklaşım ve teorilerine uygun olmadığı saptanmıştır. Kazanımların sayıca çok az olduğu, zihinsel hazırlık ve işlemlere yer verilmediği görülmüştür. Bu programla yetişen öğrencilerin konuşmada ciddi sorunlar yaşaması kaçınılmazdır. Bu durum hem öğrencilerin geleceğini hem de Türkçe öğretiminin niteliğini olumsuz etkilemektedir. Ülkemizde güncel dil öğretim yaklaşımlarına uygun Türkçe öğretim programının hazırlanması, konuşmanın zihinsel boyutlarına ağırlık verilmesi önerilmektedir.

Effect of a single lateral diffuse reflection on spatial percepts and speech intelligibility.

This study examines the influence of an early lateral reflection on spatial perceptual attributes and speech reception. To this aim, a diffuse reflection is compared with a specular one. Although diffusive surfaces have widespread applications in room acoustics design, the knowledge of the perceptual and behavioral outcomes of these surfaces has yet to be fully developed. Two experiments were conducted to investigate how the reflection type, its temporal delay, and its azimuth affect spatial percepts (source distance, width, and focus) and speech intelligibility (SI) in diffuse stationary noise. The experimental setup included ecological elements: field measurements, a speaker-like source directivity, and real flat and diffusive surfaces. The results indicate that the presence of a single diffuse reflection reduces the perceived distance of a frontal speech source and makes it clearer. SI is higher with a diffuse reflection than with a specular one. Perceptual and behavioral outcomes both depend on the angle of reflection given the frequency- and angular-dependent properties of the diffusing surface and the directivity of the speech source. The results are interpreted with reference to loudness and binaural cues and to the precedence effect. Implications of the findings for acoustic design are also discussed.

Progress in the treatment of single-sided deafness with artificial hearing aids

Single-sided deafness（SSD） is a common disease. Compared with normal hearing people, unilateral deaf patients often suffer from the loss of sound source localization ability and speech recognition in complex environments due to the lack of binaural summation effect, squelch effect, spatial demasking effect, and the presence of head shadow effect. Patients' quality of life and professional activities are always poor. Therefore, this article will review the related definitions of SSD, epidemiology, hazards, and artificial hearing aids in the application of SSD patients.

Development of Text to Speech Conversion System for Low Vision and Blind People

Around the world 285 million individuals are found to be visually challenged out of 7.4 billion populations found in a survey made by World Health Organization. These people face many problems but the major problem is reading. It is observed that they cannot read the text which is not written in braille. In the thought process of supporting them, here is a framework proposed for the visually challenged people which can perform content recognition and produce voice yield. This can assist the visually challenged people with reading any printed content and convey in speech output. A camera is utilized to capture the content from the printed content and the captured picture experiences progression of picture pre-preprocessing steps to get the content of the picture and expels the background. Characters are identified utilizing Tesseract-Optical Character recognition (OCR). The identified script is then changed into voice, utilizing open source speech synthesizer (TTS). Finally, the speech output is heard by the earphones.

Learning Efficient Representations for Fake Speech Detection

Synthetic speech or “fake speech” which matches personal vocal traits has become better and cheaper due to advances in deep learning-based speech synthesis and voice conversion approaches. This increased accessibility of synthetic speech systems and the growing misuse of them highlights the critical need to build countermeasures. Furthermore, new synthesis models evolve all the time and the efficacy of previously trained detection models on these unseen attack vectors is poor. In this paper, we focus on: 1) How can we build highly accurate, yet parameter and sample-efficient models for fake speech detection? 2) How can we rapidly adapt detection models to new sources of fake speech? We present four parameter-efficient convolutional architectures for fake speech detection with best detection F1 scores of around 97 points on a large dataset of fake and bonafide speech. We show how the fake speech detection task naturally lends itself to a novel multi-task problem further improving F1 scores for a mere 0.5% increase in model parameters. Our multi-task setting also helps in data-sparse situations, commonplace in adversarial settings. We investigate an alternative approach to the data-sparsity problem using transfer learning and show that it is possible to meet purely supervised detection performance for unseen attack vectors with as little as 6.25% of the training data. This is the first known application of transfer learning in adversarial settings for speech. Finally, we show how well our transfer learning approach adapts in an instance-efficient way to new attack vectors using the Real-Time Voice Cloning toolkit. We exceed the purely supervised detection performance (99.18 F1) with as little as 6.25% of the data.

Pemertahanan Bahasa Jawa Santri Pondok Pesantren Durrotu Ahlissunnah Waljamaah Banaran Gunungpati Semarang

AbstractJavanese language as a regional language along with the development era is starting to be forgotten. Various attempts to maintain language are carried out as a preservation effort so that Javanese language is not extinct. This study describes the problem of Javanese language retention in Durrotu Ahlissunnah Waljamaah Islamic Boarding School students. The research method used is descriptive qualitative research data source of speech by students of Islamic Boarding School Durrotu Ahlissunnah Waljamaah. The data used is a fragment of the speech of students of the Islamic Boarding School Durrotu Ahlissunnah Waljamaah. Data collection techniques using record techniques and note taking techniques. The results of the analysis can be concluded: (1) Javanese language retention Santri Islamic Boarding School Durrotu Ahlissunnah Waljamaah consists of two forms, namely (a) a form of active detention that is the use of Javanese as a whole in which no other languages are mixed, and (2) a form of passive retention that is a form of language retention characterized by the use of Javanese but containing one foreign language element or the use of other languages but there is Javanese in it.Keywords: language retention, boarding school.

Speech Sources Separation Based on Models of Interaural Parameters and Spatial Properties of Room

Speech Sources Separation Based on Models of Interaural Parameters and Spatial Properties of Room

Hybrid Source Prior Based Independent Vector Analysis for Blind Separation of Speech Signals

Blind Source Separation (BSS) application is a delinquent issue in a complex reverberant environment with changing room geometric dimensions and an increasing number of speech sources. The BSS application issue is determined by the independent component analysis that usually manipulates higher-order statistical approaches. However, the permutation between desired speech sources remains a challenging issue for BSS applications. The permutation problem is been rectified by Independent Vector Analysis (IVA) for BSS applications in the frequency domain. The performance dependency of the IVA approach solely relies on the selection of appropriate source-prior to preserve the inter-frequency dependencies between the same speech source amongst different frequency bins. Therefore, a hybrid model for the IVA method is presented, which comprises of multivariate generalized Gaussian and super-Gaussian distribution source priors to model low as well as high amplitudes speech signals. The weights of the hybrid model between multivariate Gaussian and generalized Gaussian are assigned in accordance to the energy of the observed non-stationary speech mixture signal. In the simulations, different speech mixtures are generated from various speech sources by simulated room model. The proposed approach evaluates the blind separation performance in terms of signal-to-distortion ratio (SDR) and is compared with well-known BSS methods. The results show an improvement of the proposed methodology for non-stationary speech signals over the state-of-the-art IVA models having a fixed source prior.

Deconvolved Conventional Beamforming and Adaptive Cubature Kalman Filter Based Distant Speech Perception System

A spatial-temporal processing framework integrated of speech enhancement and speech tracking is proposed in this paper for distant speech perception. First, weak speech signals are enhanced by the deconvolved conventional beamforming (DCBF) with a microphone array. By virtue of the narrow beamwidth and low sidelobes of the DCBF, the competing sources can be effectively suppressed without introducing extra speech distortion. Second, with the accurate bearing provided by the DCBF, the Cubature Kalman filter can be utilized to track the speech source of interest. By introducing a scaling factor in the current statistical motion model, a new tracking algorithm is proposed which is suitable for both maneuvering and nonmaneuvering speech sources. The introduced scaling factor can be adaptively adjusted to improve the tracking performance of the proposed algorithm for different motion models. Numerical results show that the proposed algorithm can provide better tracking performance than the conventional one. In particular, the tracking root mean square error can be reduced by half for some cases.

Successive Relative Transfer Function Identification Using Blind Oblique Projection

Distortionless speech extraction in a reverberant environment can be achieved by applying a beamforming algorithm, provided that the relative transfer functions (RTFs) of the sources and the covariance matrix of the noise are known. In this paper, the challenge of RTF identification in a multi-speaker scenario is addressed. We propose a successive RTF identification (SRI) technique, based on the sole assumption that sources do not become simultaneously active. That is, we address the challenge of estimating the RTF of a specific speech source while assuming that the RTFs of all other active sources in the environment were previously estimated in an earlier stage. The RTF of interest is identified by applying the blind oblique projection (BOP)-SRI technique. When a new speech source is identified, the BOP algorithm is applied. BOP results in a null steering toward the RTF of interest, by means of applying an oblique projection to the microphone measurements. We prove that by artificially increasing the rank of the range of the projection matrix, the RTF of interest can be identified. An experimental study is carried out to evaluate the performance of the BOP-SRI algorithm in various signal to noise ratio (SNR) and signal to interference ratio (SIR) conditions and to demonstrate its effectiveness in speech extraction tasks.

Direction and Reverberation Preserving Noise Reduction of Ambisonics Signals

Ambisonics encodes the directional information of a sound field w.r.t. a reference position in an efficient and scalable way. However, the sound field might contain undesired noise. Reducing the noise while preserving the directional distribution of all sound field components is a challenging task. Recently, the present authors proposed a direction-preserving noise reduction method for higher-order Ambisonics (HOA) signals, which, in contrast to e.g. binaural beamforming methods, yields a HOA signal at the output. In this work, we investigate the direction-preserving noise reduction method further and compare it against a beamforming-based method and the matrix multi-channel Wiener filter. Different methods to estimate the power spectral densities which are needed for the noise reduction methods are discussed. Moreover, a method to preserve the reverberation of the desired signal is proposed. In the evaluation, the discussed methods are compared for different speech sources in anechoic and reverberant conditions and different noise types.

Про деякі зміни в говірці с. Пльохів Чернігівської області

The proposed exploration represents the dynamics of the language structure elements of the dialect of Plokhiv village (Chernihiv district, Chernihiv region) according to the evidence of three time slices – the end of the XIX century (ethnographic materials of Borys Hrinchenko), middle of the XX century (data „Atlas of the Ukrainian language”) and the beginning of the XXI century (own author’s expedition records). The research was carried out with such methods as content analysis, comparative analysis, interviews, linguistic-geographical method and linguistic description. It has been noted that folklore records have long been used as sources of dialectal speech in linguistics, although rarely. They are a valuable source for identifying the dynamics of dialects; it has been proved by scientific studies of the south-eastern dialect of the Ukrainian language. However, folklore texts as a source for studying the dynamics of dialectal speech of Eastern Polissya are still out of the attention of scholars. Comparative study of the dialect of Plokhiv village in different time slices showed that a significant part of the elements of speech have not changed for a long time. At the same time, a number of language elements reduced functional activity due to the development of other forms, which began to be used in parallel. However, the tendency to preserve the structure of speech dominates, because the replacement of some forms by others is rarely recorded.

Реестр лингводидактических проблем аудирования иноязычного текста

The article presents the specifics of listening of French speech as a type of speech activity. The authors point out that in order to overcome the difficulties of listening to authentic foreign language speech, it is necessary to find out how these difficulties can be caused and identify methodological ways to overcome these difficulties. Based on the analysis of the scientific literature, a register of difficulties and problems of listening to French has been highlighted, associated with: 1) the language form of the message (phonetic, lexical, and grammatical) with the semantic content of the message; 2) the specificity of the French-speaking picture of the world and the methods of verbalization of French cultural concepts; 3) conditions of presentation of the message; 4) the sources of information and the conditions for the presentation of the message; 5) the individual characteristics of the source of speech; 6) the amount of information; 7) the individual characteristics of the recipient. The article concludes that the student’s auditing ability requires not only overcoming difficulties arising from objective features of listening as a type of speech activity and the specificity of perception and understanding of foreign language speech, but also overcoming every student’s psychological barriers that are based on their individual personality characteristics.

Effect of Directional Microphone Technology in Hearing Aids on Neural Correlates of Listening and Memory Effort: An Electroencephalographic Study.

The aim of the study was to compare the effect of different spatial noise-processing algorithms in hearing aids on listening effort and memory effort on a subjective, behavioral, and neurophysiological level using electroencephalography (EEG). Two types of directional microphone (DM) technologies for spatial noise processing were chosen: one with a wide directionality (wide DM) and another with a narrower directionality (narrow DM) to accentuate the speech source. Participants with a severe hearing loss were fitted with hearing aids and participated in two EEG experiments. In the first one, participants listened to sentences in cafeteria noise and were asked to rate the experienced listening effort. The second EEG experiment was a listening span task during which participants had to repeat sentence material and then recall the final words of the last four sentences. Subjective listening effort was lower with narrow than wide DM and EEG alpha power was reduced for the narrow DM. The results of the listening span task indicated a reduction in experienced memory effort and better memory performance. During the memory retention phase, EEG alpha level for the narrow relative to the wide DM was reduced. This effect was more pronounced during linguistically difficult sentences. This study extends previous findings, as it reveals a benefit for narrow DM in terms of cognitive performance and memory effort also on a neural level, and when speech intelligibility is almost 100%. Together, this indicates that a narrow and focused DM allows for a more efficient neurocognitive processing than a wide DM.

Spatial Coherence-Aware Multi-Channel Wind Noise Reduction

Outdoor recording is particularly challenging in the presence of wind, which induces highly non-stationary noise in the microphone signals. To enhance a desired signal, e.g., speech, a dedicated noise reduction processing is required. The reduction is usually performed by estimating an unknown set of parameters, e.g., the noise and the speech power spectral densities (PSDs). In contrast to the commonly used assumption of uncorrelated wind noise in multi-channel recordings, we assume the spatial correlation of wind noise contributions to be non-zero at low frequencies when closely-spaced microphones are employed. In our earlier work, we assumed that the spatial coherence was known, i.e., given by a model which depends on the free-field speed and direction of the air stream. As these quantities are unknown in practice, we propose in this work a method to recursively estimate the spatial coherence matrix based on the microphone observations. In addition, we prove the equivalence of two recently developed noise PSD estimation methods when uncorrelated wind noise is assumed, and we propose an approximation of both estimators which is independent of the propagation vector of the speech source at sufficiently low frequency and for a small inter-microphone distance. An evaluation in terms of improvements in speech quality, signal-to-noise ratio and intelligibility is carried out using both simulated and measured wind noise samples.

Spectral Flux-Based Convolutional Neural Network Architecture for Speech Source Localization and Its Real-Time Implementation.

In this article, we present a real-time convolutional neural network (CNN)-based Speech source localization (SSL) algorithm that is robust to realistic background acoustic conditions (noise and reverberation). We have implemented and tested the proposed method on a prototype (Raspberry Pi) for real-time operation. We have used the combination of the imaginary-real coefficients of the short-time Fourier transform (STFT) and Spectral Flux (SF) with delay-and-sum (DAS) beamforming as the input feature. We have trained the CNN model using noisy speech recordings collected from different rooms and inference on an unseen room. We provide quantitative comparison with five other previously published SSL algorithms under several realistic noisy conditions, and show significant improvements by incorporating the Spectral Flux (SF) with beamforming as an additional feature to learn temporal variation in speech spectra. We perform real-time inferencing of our CNN model on the prototyped platform with low latency (21 milliseconds (ms) per frame with a frame length of 30 ms) and high accuracy (i.e. 89.68% under Babble noise condition at 5dB SNR). Lastly, we provide a detailed explanation of real-time implementation and on-device performance (including peak power consumption metrics) that sets this work apart from previously published works. This work has several notable implications for improving the audio-processing algorithms for portable battery-operated Smart loudspeakers and hearing improvement (HI) devices.

Speech Source Separation Using Variational Autoencoder and Bandpass Filter

Speech source separation is essential for speech-related applications because this process enhances the input speech signal for the main processing model. Most of the current approaches for this task focus on separating the speech of commonly high-frequency noises or a particular background sound. They cannot clear the signals which intersect with the human speech in its frequency range. To deal with this problem, we propose a hybrid approach combining a variational autoencoder (VAE) and a bandpass filter (BPF). This method can extract and enhance the speech signal in the mixture of many elements such as speech signal, the high-frequency noises, and many kinds of different background sounds which interfere with the speech sound. Experimental results showed that our model can extract effectively the speech signal with 15.02 dB in Signal to Interference Ratio (SIR) and 12.99 dB in Signal to Distortion Ratio (SDR). On the other hand, we can adjust the passband to identify the range of frequency at the output signal to apply for a particular application like gender classification.

The Root of Indian Communication Theory in the Ṛig Veda: Practical Vedānta

What is the source of the power of speech and eloquence and fulfillment in life? Though communication and rhetoric departments in most Indian universities have been focusing their teaching and research agendas on Western models, a growing body of scholarship is developing communication theory that approaches the big questions from an Indian perspective, drawing on traditional sources (Adhikary, 2014), which claim Veda as their ultimate source. This paper explores the Vedic worldview on speech and communication proclaimed in the Ṛicho Akśare verse of the Ṛig Veda, and others, drawing on sage Bhartṛhari (c. 450-500 CE), Maharishi Mahesh Yogi (1975; 1971), and Sanskrit scholars of the philosophy of language, who reference these hymns. They describe a Vedic cosmology of speech that bears striking resemblance to the universe according to string theory of quantum physics. The science serves to corroborate the premise of Vedic levels-of-speech theory that the universe is structured and governed by laws of nature/language of nature from within an unmanifest unified field of all the laws of nature, which Ṛig Veda 1.164 calls Parā and identifies as consciousness. This inquiry helps to illuminate how speech is Brahman, the source and goal of understanding, eloquence, and fulfillment. The Vedic texts enjoin the sanātana dharma of yoga, opening awareness to the transcendental source of speech. I conclude that Vedic communication theory embedded in the hymns is integral to practical Vedanta. As Muktitkā Upaniṣad 1.9 proclaims: “As oil is present in a sesame seed, so Vedānta is present in the Veda.”