Speech Information Processing Research Articles

Language networks of normal-hearing infants exhibit topological differences between resting and steady states: An fNIRS functional connectivity study.

Task-related studies have consistently reported that listening to speech sounds activate the temporal and prefrontal regions of the brain. However, it is not well understood how functional organization of auditory and language networks differ when processing speech sounds from its resting state form. The knowledge of language network organization in typically developing infants could serve as an important biomarker to understand network-level disruptions expected in infants with hearing impairment. We hypothesized that topological differences of language networks can be characterized using functional connectivity measures in two experimental conditions (1) complete silence (resting) and (2) in response to repetitive continuous speech sounds (steady). Thirty normal-hearing infants (14 males and 16 females, age: 7.8 ± 4.8 months) were recruited in this study. Brain activity was recorded from bilateral temporal and prefrontal regions associated with speech and language processing for two experimental conditions: resting and steady states. Topological differences of functional language networks were characterized using graph theoretical analysis. The normalized global efficiency and clustering coefficient were used as measures of functional integration and segregation, respectively. We found that overall, language networks of infants demonstrate the economic small-world organization in both resting and steady states. Moreover, language networks exhibited significantly higher functional integration and significantly lower functional segregation in resting state compared to steady state. A secondary analysis that investigated developmental effects of infants aged 6-months or below and above 6-months revealed that such topological differences in functional integration and segregation across resting and steady states can be reliably detected after the first 6-months of life. The higher functional integration observed in resting state suggests that language networks of infants facilitate more efficient parallel information processing across distributed language regions in the absence of speech stimuli. Moreover, higher functional segregation in steady state indicates that the speech information processing occurs within densely interconnected specialized regions in the language network.

Dynamic causal modeling analysis reveals the modulation of motor cortex and integration in superior temporal gyrus during multisensory speech perception.

The online version contains supplementary material available at 10.1007/s11571-023-09945-z.

Вопрос о становлении классической традиции иранской музыки в аспекте проблемы языка и мышления

The article examines the historical prerequisites for the formation of the classical tradition of Iranian music of the Achaemenid and Sassanid dynasties period (6th century BCE — the first third of the 7th century CE) in a linguistic environment with the dominant function of the Aramaic language (Old Aramaic, Imperial Koine, Syriac/ New Aramaic dialects) at the sites of the “royal way”, where the local musical heritage of Indo-Iranian tribes melted in the phonosphere of the cosmopolitan palace vocal and instrumental music-making of ancient civilizations. The author of the article formulates the hypothesis that (1) when working with the concepts “Indo-European language family” or “Semitic language family”, we describe the contrasting mechanisms of speech information processing and transmission that (2) appear universal for different spheres of intellectual activity, including music, where they (3) are expressed in two formulae of the musical process I -- m -- t (B. Asafyev) and I -- t = m—temporalityy (G. Shamilli) (3) that is sensitive to certain patterns of thinking and (4) is fixed in the gestalt. Therefore, to think of derivation mechanisms of verbal and musical speech as unified means to provide a gestalt morphosyntactic description of structures from an interdisciplinary perspective, relying on (1) the fact of the linguistic community of cultures over a long historical period and (2) the assumption about the ability of the oral musical tradition to retain one gestalt or another with a change of the language domain.

Research of application of DFT-modulated filter bank in systems with significant spectral component amplification

The purpose of this article is to investigate the application of DFT-modulated filter bank in systems with significant spectral component amplification like hearing aid. There is a description of analysis / synthesis method based on short-time Fourier transform (STFT), which is used in most systems of speech information processing. It is shown that DFT-modulated filter bank is a generalization of STFT-method. In analysis / synthesis system based on DFT-modulated filter bank, the input signal is divided into subbands, passing through the analysis filter bank then each subband is amplified and the last step is to reconstruct the signal with synthesis filter bank. However, in digital systems with significant spectral component amplification, the resulting signal is distorted after reconstruction because of amplification factor difference in each subband. The article provides expressions for the distortion and the aliasing functions, allowing to estimate the distortion value, which appears in analysis / synthesis system of DFT-modulated filter bank. Efficient algorithms for calculating the distortion and the aliasing functions are also offered. In future it is planning to develop a procedure for optimizing the DFT-modulated filter bank based on the proposed efficient algorithms for calculating distortion and spectral aliasing in the filter bank.

ПРИМЕНЕНИЕ ФАЗОВЫХ ХАРАКТЕРИСТИК ГОЛОСОВЫХ ВОКАЛИЗМОВ В РЕШЕНИИ ЗАДАЧ ЗАЩИТЫ РЕЧЕВОЙ ИНФОРМАЦИИ

At present, due to its convenience, speech technologies are gaining increasing popularity, including in relation to security systems. However, due to possible security problems, the areas in which this technology is extremely limited still remain. The aim of this work is to clarify the field of application of the phase characteristics of voice vocalisms in solving various problems of protection and processing of speech information. Methods. The methods of computer and mathematical modeling, digital processing of signals and images are used. Results . It is proposed to use the phase characteristics of speech in various applications, including security problems such as a reliable identification. Conclusion . The major characteristics of speech signals are presented. The Hilbert model of the speech signal is considered, as well as the model of the sinusoidal description of the speech signal by MacAuel and Quatieri. The areas of practical application of the phase characteristics of voice vocalisms (including the introduction of biometric information about the speaker into the phaseogram) are described.

English Grammar Detection Based on LSTM-CRF Machine Learning Model.

Deep learning and neural network have been widely used in the field of speech, vocabulary, text, pictures, and other information processing fields, which has achieved excellent research results. Neural network algorithm and prediction model were used in this paper for the study and exploration of English grammar. Aiming at the application requirements of English grammar accuracy and standardization, we proposed a machine learning model based on LSTM-CRF to detect and analyze English grammar. This paper briefly summarized the development trend of deep learning and neural network algorithm and designed the structure pattern of radial basis function neural network in grammar semantic detection and analysis on the basis of deep learning artificial neural network theory. Based on the morphological features of English grammar, a grammar database was established according to the rules of English word segmentation. In this paper, we proposed an improved conditional random field CRF (Conditional Random Field) network model based on LSTM (Long Short-Term Memory) neural network. It can improve the problem that the traditional machine learning model relies on feature point selection in English grammar detection. The machine learning model based on LSTM-CRF was used to recognize English grammar text entities. The results show that the English grammar detection system based on the LSTM-CRF model can simplify the process structure in the recognition process, reduce the unnecessary operation cycle, and improve the overall detection accuracy.

Diagnostic Biomarker Exploration of Autistic Patients With Different Ages and Different Verbal Intelligence Quotients Based on Random Forest Model

As a neurodevelopmental disorder with complex pathogenesis, the existing diagnostic methods of autism still only rely on the scale method. In recent years, there have been some methods using machine learning to classify Autism Spectrum Disorders (ASD), and achieved good accuracy. However, the generalization of these models is poor, and the conclusions are inconsistent. The main reason is that most of them use single site dataset or private dataset for analysis, which will lead to one-sided conclusion. They did not analyze the phenotypic features of the dataset, such as handedness, gender, and age. In order to make the obtained brain diagnostic biomarkers of ASD more universal and generalized, instead of analyzing the dataset from a single site, the whole dataset is divided into subgroups according to age and Verbal Intelligence Quotient (VIQ), and then each subgroup is classified and analyzed by Random Forest (RF) model. The experimental results show that if all male subjects are used for classification, the accuracy of classification can only reach about 55%. By using the proposed grouping method and RF model, the classification accuracy for different subgroups will be improved by 3% ~ 17%. Through the analysis of the importance and difference of the features in each subgroup, we can find that the features obtained in the above experiments are closely related to the functions of speech, emotion, auditory and visual information processing. This may partly explain why ASD patients have speech, social disorder, repetitive behavior and narrow interests. The classification methods proposed and diagnostic biomarkers obtained in this paper may provide some reference for the clinical diagnosis and early treatment of ASD.

Design of Marine Two-Way Voice Communication System Based on Human-Computer Interaction

Zhu, T. and Zhang, F., 2020. Design of marine two-way voice communication system based on human-computer interaction. In: Malvarez, G. and Navas, F. (eds.), Global Coastal Issues of 2020. Journal of Coastal Research, Special Issue No. 95, pp. 1389-1394. Coconut Creek (Florida), ISSN 0749-0208.In order to improve the long-distance transmission and speech receiving and receiving ability of marine bidirectional voice communication, the marine bi-directional voice communication transmission technology based on human-computer interaction is proposed in this paper. The channel equalization model of marine bi-directional speech communication is designed to modulate the marine bi-directional speech communication signal in the space-disguised coherent multi-path channel of marine bi-directional speech information transmission, and the linear equalization controller is used to filter the interference. The fractionally spaced equalizer is designed to deal with the anti-interference of the marine bidirectional speech communication signal, to improve the real-time performance of the marine bi-directional speech transmission, and to design the receiver at the output terminal to realize the accurate feedback of the marine bi-directional speech communication signal. The hardware modularization design of marine bi-directional speech communication system based on man-machine interaction model is carried out. The main function modules of the system include data receiving module, speech information processing module, human-computer interaction module and receiver conversion module. The simulation results show that the proposed method has low bit error rate (BER) for long-distance transmission of marine bi-directional speech communication signals, and can realize real-time monitoring and voice communication of marine bi-directional speech communication.

Специфика декодирования логико-грамматических конструкций языка младшими школьниками с недоразвитием речи

The article presents the results obtained in the course of a scientific study of decoding processes by younger schoolchildren with speech underdevelopment of the logical and grammatical structures of the language, which include phraseological phrases. Covering a wide range of psycholinguistic difficulties, the author describes the speech-language mechanism, complex in its organization and semantic content, which is involved in the decoding of text messages. At the level of the facts of the ascertaining experiment, a series of tasks is described, which allows to draw conclusions about the presence of specific difficulties caused by the insufficient implementation of cognitive processes during the operation of verbal material, the desemantization of speech messages, the low level of knowledge of decoding ability, which provides information processing of speech information of varying degrees of complexity.

Language Recognition Method of Convolutional Neural Network Based on Spectrogram

Language recognition is an important branch of speech technology. As a front-end technology of speech information processing, higher recognition accuracy is required. It is found through research that there are obvious differences between the language maps of different languages, which can be used for language identification. This paper uses a convolutional neural network as a classification model, and compares the language recognition effects of traditional language recognition features and spectrogram features on the five language recognition tasks of Chinese, Japanese, Vietnamese, Russian, and Spanish through experiments. The best effect is the ivector feature, and the spectrogram feature has a higher F value than the low-dimensional ivector feature.

An overview of speech endpoint detection algorithms

Speech endpoint detection is an important part of modern speech information processing technology. The success of endpoint detection directly improves the performance and quality of speech coding, speech recognition, speech synthesis and human interaction. The robustness and detection accuracy of algorithms have always been hot topics for many scholars in the condition of low Signal-to-Noise Ratio (SNR) and complex noise. In this paper, we aim to provide an overview of the state-of-the-art in time domain, frequency domain and cepstrum domain for speech endpoint detection algorithms and to cast a glance at the challenges for future research.

Research Progress Discussion on Deep Language Signal and Information Processing

This article focus on the progress discussion of deep speech signal and information processing. It will combine theory with practice, and deeply analyze the research progress of speech recognition technology, speech blending technique, and speech intensified technique. Based on the big data and cloud computing, the conclusion of valuable information can be effectively identified through deep speech signal and information processing technology. Hope it will help the relevant units.

Auditory and Somatosensory Interaction in Speech Perception in Children and Adults.

Multisensory integration (MSI) allows us to link sensory cues from multiple sources and plays a crucial role in speech development. However, it is not clear whether humans have an innate ability or whether repeated sensory input while the brain is maturing leads to efficient integration of sensory information in speech. We investigated the integration of auditory and somatosensory information in speech processing in a bimodal perceptual task in 15 young adults (age 19–30) and 14 children (age 5–6). The participants were asked to identify if the perceived target was the sound /e/ or /ø/. Half of the stimuli were presented under a unimodal condition with only auditory input. The other stimuli were presented under a bimodal condition with both auditory input and somatosensory input consisting of facial skin stretches provided by a robotic device, which mimics the articulation of the vowel /e/. The results indicate that the effect of somatosensory information on sound categorization was larger in adults than in children. This suggests that integration of auditory and somatosensory information evolves throughout the course of development.

Time-compression thresholds for Mandarin sentences in normal-hearing and cochlear implant listeners

Faster speech may facilitate more efficient communication, but if speech is too fast it becomes unintelligible. The maximum speeds at which Mandarin words were intelligible in a sentence context were quantified for normal hearing (NH) and cochlear implant (CI) listeners by measuring time-compression thresholds (TCTs) in an adaptive staircase procedure. In Experiment 1, both original and CI-vocoded time-compressed speech from the MSP (Mandarin speech perception) and MHINT (Mandarin hearing in noise test) corpora was presented to 10 NH subjects over headphones. In Experiment 2, original time-compressed speech was presented to 10 CI subjects and another 10 NH subjects through a loudspeaker in a soundproof room. Sentences were time-compressed without changing their spectral profile, and were presented up to three times within a single trial. At the end of each trial, the number of correctly identified words in the sentence was scored. A 50%-word recognition threshold was tracked in the psychophysical procedure. The observed median TCTs were very similar for MSP and MHINT speech. For NH listeners, median TCTs were around 16.7 syllables/s for normal speech, and 11.8 and 8.6 syllables/s respectively for 8 and 4 channel tone-carrier vocoded speech. For CI listeners, TCTs were only around 6.8 syllables/s. The interquartile range of the TCTs within each cohort was smaller than 3.0 syllables/s. Speech reception thresholds in noise were also measured in Experiment 2, and were found to be strongly correlated with TCTs for CI listeners. In conclusion, the Mandarin sentence TCTs were around 16.7 syllables/s for most NH subjects, but rarely faster than 10.0 syllables/s for CI listeners, which quantitatively illustrated upper limits of fast speech information processing with CIs.

Informational Masking Effects on Neural Encoding of Stimulus Onset and Acoustic Change.

Recent investigations using cortical auditory evoked potentials have shown masker-dependent effects on sensory cortical processing of speech information. Background noise maskers consisting of other people talking are particularly difficult for speech recognition. Behavioral studies have related this to perceptual masking, or informational masking, beyond just the overlap of the masker and target at the auditory periphery. The aim of the present study was to use cortical auditory evoked potentials, to examine how maskers (i.e., continuous speech-shaped noise [SSN] and multi-talker babble) affect the cortical sensory encoding of speech information at an obligatory level of processing. Specifically, cortical responses to vowel onset and formant change were recorded under different background noise conditions presumed to represent varying amounts of energetic or informational masking. The hypothesis was, that even at this obligatory cortical level of sensory processing, we would observe larger effects on the amplitude and latency of the onset and change components as the amount of informational masking increased across background noise conditions. Onset and change responses were recorded to a vowel change from /u-i/ in young adults under four conditions: quiet, continuous SSN, eight-talker (8T) babble, and two-talker (2T) babble. Repeated measures analyses by noise condition were conducted on amplitude, latency, and response area measurements to determine the differential effects of these noise conditions, designed to represent increasing and varying levels of informational and energetic masking, on cortical neural representation of a vowel onset and acoustic change response waveforms. All noise conditions significantly reduced onset N1 and P2 amplitudes, onset N1-P2 peak to peak amplitudes, as well as both onset and change response area compared with quiet conditions. Further, all amplitude and area measures were significantly reduced for the two babble conditions compared with continuous SSN. However, there were no significant differences in peak amplitude or area for either onset or change responses between the two different babble conditions (eight versus two talkers). Mean latencies for all onset peaks were delayed for noise conditions compared with quiet. However, in contrast to the amplitude and area results, differences in peak latency between SSN and the babble conditions did not reach statistical significance. These results support the idea that while background noise maskers generally reduce amplitude and increase latency of speech-sound evoked cortical responses, the type of masking has a significant influence. Speech babble maskers (eight talkers and two talkers) have a larger effect on the obligatory cortical response to speech sound onset and change compared with purely energetic continuous SSN maskers, which may be attributed to informational masking effects. Neither the neural responses to the onset nor the vowel change, however, were sensitive to the hypothesized increase in the amount of informational masking between speech babble maskers with two talkers compared with eight talkers.

Music, speech, and temporal information processing

Music and speech share many acoustic characteristics. Both utilize dynamic acoustic cues (e.g., frequency, time, intensity) to convey information and emotion. We focus on temporal information processing at a slower time scale (e.g., musical timing, speech rhythm) to examine connections between music and speech: (1) transfer effects, (2) neural mechanisms and (3) theoretical frameworks. I will focus on early infancy. We observed a transfer effect from music experience to both music and speech processing in 9-month-old infants (Zhao & Kuhl, 2016). Infants were randomly assigned to either a 12-session music intervention that focused on triple meter (waltz) or a 12-session control group (no music) during the “sensitive period” for phonetic learning. The music intervention group exhibited enhanced neural detection of both mistimed musical beats and speech syllables, quantified by the mismatch response (MMR) measured with magnetoencephalography (MEG). The effects were observed in auditory and prefrontal regions of the cortex. Following the "predictive coding" and "dynamic attending" theories, I will discuss a current two-pronged approach that can extend this research to understand the effects of music on infants’ higher-level cognitive skills as well as their early stage sensory encoding of sounds, and implications of these findings for theories of language development.

Speech-on-speech masking and psychotic symptoms in schizophrenia

People with schizophrenia have impairments of target-speech recognition (TSR) in noisy environments with multiple people talking. This study investigated whether the TSR impairment in schizophrenia is associated with their impaired auditory working memory or certain psychotic symptoms. Thirty participants with schizophrenia (mean age = 35.2 ± 12.7 years) and 30 demographics-matched healthy controls (mean age = 32.9 ± 10.9 years) were tested for their TSR against a two-talker-speech masker. Auditory working memory and memory capacity were evaluated using the Paced Auditory Serial Addition Test (PASAT) and Digit Span Test. Psychotic symptoms were evaluated using the Positive and Negative Syndrome Scale (PANSS). The results showed that participants with schizophrenia had higher TSR threshold (i.e., poorer TSR performance) and poorer PASAT scores than their healthy controls. Moreover, positive correlations (with age, sex, educational years, ill-duration, and dosage of antipsychotics controlled as covariates) were revealed between the TSR threshold and the PANSS-positive syndrome (especially delusion), and between the TSR threshold and the PANSS-negative syndrome (especially lack of spontaneity in speech and passive-apathetic-social withdraw). However, neither the PASAT nor the forward digit span exhibited significant correlations with the TSR. This study provides evidence that the TSR impairment (i.e., augmented vulnerability to informational speech masking), which reflects disorganization of speech information processing (inability in either inhibiting unrelated speech signals or capturing the wanted speech signals), is specifically associated with the severity of delusion, poverty of speech, and hypobulia, suggesting the potential value of the TSR impairment used for predicting certain core symptoms of schizophrenia.

Impact of group-singing on older adult health in senior living communities: A pilot study

PurposeParticipating in a group-singing program may be beneficial to healthy aging through engaging in active music-making activities and breathing exercises. The purpose of this study was to assess the feasibility, acceptability, and impact of a 12-week group singing program on cognitive function, lung health and quality of life (QoL) of older adults. Materials and methodsA pre and post-test quasi-experimental design evaluated the impact of a group-singing program on older adult health. The intervention consisted of pre-singing exercises, song-singing and learning, and socialization. Classes were 75 min/week for 12 weeks. Inclusion criteria were age ≥60, no self-reported diagnosis of dementia, and able to hear conversations within 2 feet. Participants were recruited from 3 senior living communities. Outcome measures included cognition, lung function, QoL, and program feasibility and acceptability. A paired t-test with 2-sided alpha level at 0.05 was used to test the null hypotheses. ResultsWe enrolled 49 participants (mean age 83.6). Forty-two (86%) completed the posttests and exit survey. At the 12th week there was significant improvement in phonological (p < 0.0001) and animal (p = 0.0004) semantic Verbal Fluency Tests, immediate Word Recall Test (p < 0.0001), Maximum Inspiratory Pressure (p = 0.0001), Maximum Expiratory Pressure (p < 0.0001), and in-session oxygen saturation (p = 0.03). Program satisfaction was rated, on average, 9 on a 10-point scale. ConclusionA group-singing program with deep breathing training and song-learning may promote memory, language, speech information processing, executive function, and respiratory muscle strength in older adults. The program was feasible and well-accepted. A clinical trial with a larger sample is indicated.

Utilizing Speech Information Processing Technology in Programming Education

Utilizing Speech Information Processing Technology in Programming Education

It doesn't matter what you say: FMRI correlates of voice learning and recognition independent of speech content

Listeners can recognize newly learned voices from previously unheard utterances, suggesting the acquisition of high-level speech-invariant voice representations during learning. Using functional magnetic resonance imaging (fMRI) we investigated the anatomical basis underlying the acquisition of voice representations for unfamiliar speakers independent of speech, and their subsequent recognition among novel voices. Specifically, listeners studied voices of unfamiliar speakers uttering short sentences and subsequently classified studied and novel voices as “old” or “new” in a recognition test. To investigate “pure” voice learning, i.e., independent of sentence meaning, we presented German sentence stimuli to non-German speaking listeners. To disentangle stimulus-invariant and stimulus-dependent learning, during the test phase we contrasted a “same sentence” condition in which listeners heard speakers repeating the sentences from the preceding study phase, with a “different sentence” condition. Voice recognition performance was above chance in both conditions although, as expected, performance was higher for same than for different sentences. During study phases activity in the left inferior frontal gyrus (IFG) was related to subsequent voice recognition performance and same versus different sentence condition, suggesting an involvement of the left IFG in the interactive processing of speaker and speech information during learning. Importantly, at test reduced activation for voices correctly classified as “old” compared to “new” emerged in a network of brain areas including temporal voice areas (TVAs) of the right posterior superior temporal gyrus (pSTG), as well as the right inferior/middle frontal gyrus (IFG/MFG), the right medial frontal gyrus, and the left caudate. This effect of voice novelty did not interact with sentence condition, suggesting a role of temporal voice-selective areas and extra-temporal areas in the explicit recognition of learned voice identity, independent of speech content.