High-level Speech Research Articles

Центральные слуховые расстройства: причины, симптомы и способы преодоления дефицита в условиях учебного процесса

<p style="text-align: justify;">Central Auditory Processing Disorders (CAPD) are impairments in the ability to recognize sounds, localize their sources, and/or determine their identity and meaning due to functional impairments of the central auditory system. CAPD is observed in individuals with normal hearing levels and is not the result of high-level speech or cognitive impairment. This disorder is not well known to Russian specialists in the field of correctional psychology, while according to foreign sources it is observed in 2—7% of schoolchildren and is often combined with other developmental disorders (dyslexia, speech developmental disorders, ASD, ADHD). Here we provide a brief review of CAPD: its causes, symptoms, and approaches to diagnosis. In particular, we draw the readers' attention to the insufficient development of tools and practices for diagnosing and treating CAPD in Russia. Next, we review current approaches to the correction of CAPD, with special emphasis on assistive technologies that improve the signal-to-noise ratio (FM systems). Evidence in the literature indicates that in addition to the immediate effect (improved speech understanding), prolonged use of FM systems has a positive long-term effect based on mechanisms of neural plasticity.</p>

Jarvis: AI Virtual Assistant

"Jarvis" was the protagonist of Tony's Stark starring in the movie Iron Man. Unlike the first jokes when Jarvis was Stark's servant, Jarvis' movie version is a smart computer that communicates critical issues, monitors his family, and helps build and organize his hero's suit. In this project, Jarvis is a Digital Health Assistant who uses social media i.e., voice that creates two-way communication between people and his personnel Computer, performing various tasks such as calling and chatting on WhatsApp, taking notes, locating, etc. In our project, we use the word just as communication means that Jarvis is a Speech recognition program. The concept of speech technology combines two technologies: Synthesizer and vision. Speech synthesizer captures input and generates audio streams as output this is achieved using the pyttsx3 library accompanied by Speech function. Speech recognition. On the other hand, he did the opposite. It treats audio streaming as input and thus converts it into text writing and this output uses the Speech Recognition library accompanied by the TakeCommand function. The word is a symbol of endless knowledge. Direct analysis and integration of complex voice signal is due to the vast amount of information contained in the signal. Therefore, digital signal processes such as Feature Visibility and Feature Matching are introduced to represent the voice signal. For this project, we are directly using a speech engine that uses an output element such as Mel scaled frequency cepstral. The mel scale frequency cepstral coefficients (MFCCs) obtained from the fourier transform and filter bank analysis are probably the most widely used in high-level speech recognition systems. We aim to create more jobs that can help people in their daily lives and reduce their efforts as in this project we have introduced Chrome and Youtube Automation. In our test, we checked that all of this functionality works well. We test this on 2 speakers (1 female and 1 male) to find the exact motive for the speculation.

Breaking down the cocktail party: Attentional modulation of cerebral audiovisual speech processing

Recent studies utilizing electrophysiological speech envelope reconstruction have sparked renewed interest in the cocktail party effect by showing that auditory neurons entrain to selectively attended speech. Yet, the neural networks of attention to speech in naturalistic audiovisual settings with multiple sound sources remain poorly understood. We collected functional brain imaging data while participants viewed audiovisual video clips of lifelike dialogues with concurrent distracting speech in the background. Dialogues were presented in a full-factorial design, comprising task (listen to the dialogues vs. ignore them), audiovisual quality and semantic predictability. We used univariate analyses in combination with multivariate pattern analysis (MVPA) to study modulations of brain activity related to attentive processing of audiovisual speech. We found attentive speech processing to cause distinct spatiotemporal modulation profiles in distributed cortical areas including sensory and frontal-control networks. Semantic coherence modulated attention-related activation patterns in the earliest stages of auditory cortical processing, suggesting that the auditory cortex is involved in high-level speech processing. Our results corroborate views that emphasize the dynamic nature of attention, with task-specificity and context as cornerstones of the underlying neuro-cognitive mechanisms.

Corrigendum: Training on Movement Figure-Ground Discrimination Remediates Low-Level Visual Timing Deficits in the Dorsal Stream, Improving High-Level Cognitive Functioning, Including Attention, Reading Fluency, and Working Memory.

[This corrects the article DOI: 10.3389/fnhum.2017.00236.].

Uncovering mental representations of smiled speech using reverse correlation.

Which spectral cues underlie the perceptual processing of smiles in speech? Here, the question was addressed using reverse-correlation in the case of the isolated vowel [a]. Listeners were presented with hundreds of pairs of utterances with randomly manipulated spectral characteristics and were asked to indicate, in each pair, which was the most smiling. The analyses revealed that they relied on robust spectral representations that specifically encoded vowel's formants. These findings demonstrate the causal role played by formants in the perception of smile. Overall, this paper suggests a general method to estimate the spectral bases of high-level (e.g., emotional/social/paralinguistic) speech representations.

Sleep Disrupts High-Level Speech Parsing Despite Significant Basic Auditory Processing.

The extent to which the sleeping brain processes sensory information remains unclear. This is particularly true for continuous and complex stimuli such as speech, in which information is organized into hierarchically embedded structures. Recently, novel metrics for assessing the neural representation of continuous speech have been developed using noninvasive brain recordings that have thus far only been tested during wakefulness. Here we investigated, for the first time, the sleeping brain's capacity to process continuous speech at different hierarchical levels using a newly developed Concurrent Hierarchical Tracking (CHT) approach that allows monitoring the neural representation and processing-depth of continuous speech online. Speech sequences were compiled with syllables, words, phrases, and sentences occurring at fixed time intervals such that different linguistic levels correspond to distinct frequencies. This enabled us to distinguish their neural signatures in brain activity. We compared the neural tracking of intelligible versus unintelligible (scrambled and foreign) speech across states of wakefulness and sleep using high-density EEG in humans. We found that neural tracking of stimulus acoustics was comparable across wakefulness and sleep and similar across all conditions regardless of speech intelligibility. In contrast, neural tracking of higher-order linguistic constructs (words, phrases, and sentences) was only observed for intelligible speech during wakefulness and could not be detected at all during nonrapid eye movement or rapid eye movement sleep. These results suggest that, whereas low-level auditory processing is relatively preserved during sleep, higher-level hierarchical linguistic parsing is severely disrupted, thereby revealing the capacity and limits of language processing during sleep.SIGNIFICANCE STATEMENT Despite the persistence of some sensory processing during sleep, it is unclear whether high-level cognitive processes such as speech parsing are also preserved. We used a novel approach for studying the depth of speech processing across wakefulness and sleep while tracking neuronal activity with EEG. We found that responses to the auditory sound stream remained intact; however, the sleeping brain did not show signs of hierarchical parsing of the continuous stream of syllables into words, phrases, and sentences. The results suggest that sleep imposes a functional barrier between basic sensory processing and high-level cognitive processing. This paradigm also holds promise for studying residual cognitive abilities in a wide array of unresponsive states.

Training on Movement Figure-Ground Discrimination Remediates Low-Level Visual Timing Deficits in the Dorsal Stream, Improving High-Level Cognitive Functioning, Including Attention, Reading Fluency, and Working Memory.

The purpose of this study was to determine whether neurotraining to discriminate a moving test pattern relative to a stationary background, figure-ground discrimination, improves vision and cognitive functioning in dyslexics, as well as typically-developing normal students. We predict that improving the speed and sensitivity of figure-ground movement discrimination (PATH to Reading neurotraining) acts to remediate visual timing deficits in the dorsal stream, thereby improving processing speed, reading fluency, and the executive control functions of attention and working memory in both dyslexic and normal students who had PATH neurotraining more than in those students who had no neurotraining. This prediction was evaluated by measuring whether dyslexic and normal students improved on standardized tests of cognitive skills following neurotraining exercises, more than following computer-based guided reading (Raz-Kids (RK)). The neurotraining used in this study was visually-based training designed to improve magnocellular function at both low and high levels in the dorsal stream: the input to the executive control networks coding working memory and attention. This approach represents a paradigm shift from the phonologically-based treatment for dyslexia, which concentrates on high-level speech and reading areas. This randomized controlled-validation study was conducted by training the entire second and third grade classrooms (42 students) for 30 min twice a week before guided reading. Standardized tests were administered at the beginning and end of 12-weeks of intervention training to evaluate improvements in academic skills. Only movement-discrimination training remediated both low-level visual timing deficits and high-level cognitive functioning, including selective and sustained attention, reading fluency and working memory for both dyslexic and normal students. Remediating visual timing deficits in the dorsal stream revealed the causal role of visual movement discrimination training in improving high-level cognitive functions such as attention, reading acquisition and working memory. This study supports the hypothesis that faulty timing in synchronizing the activity of magnocellular with parvocellular visual pathways in the dorsal stream is a fundamental cause of dyslexia and being at-risk for reading problems in normal students, and argues against the assumption that reading deficiencies in dyslexia are caused by phonological or language deficits, requiring a paradigm shift from phonologically-based treatment of dyslexia to a visually-based treatment. This study shows that visual movement-discrimination can be used not only to diagnose dyslexia early, but also for its successful treatment, so that reading problems do not prevent children from readily learning.

Assessment of the Speech Intelligibility Performance of Post Lingual Cochlear Implant Users at Different Signal-to-Noise Ratios Using the Turkish Matrix Test.

Spoken word recognition and speech perception tests in quiet are being used as a routine in assessment of the benefit which children and adult cochlear implant users receive from their devices. Cochlear implant users generally demonstrate high level performances in these test materials as they are able to achieve high level speech perception ability in quiet situations. Although these test materials provide valuable information regarding Cochlear Implant (CI) users' performances in optimal listening conditions, they do not give realistic information regarding performances in adverse listening conditions, which is the case in the everyday environment. The aim of this study was to assess the speech intelligibility performance of post lingual CI users in the presence of noise at different signal-to-noise ratio with the Matrix Test developed for Turkish language. Cross-sectional study. The thirty post lingual implant user adult subjects, who had been using implants for a minimum of one year, were evaluated with Turkish Matrix test. Subjects' speech intelligibility was measured using the adaptive and non-adaptive Matrix Test in quiet and noisy environments. The results of the study show a correlation between Pure Tone Average (PTA) values of the subjects and Matrix test Speech Reception Threshold (SRT) values in the quiet. Hence, it is possible to asses PTA values of CI users using the Matrix Test also. However, no correlations were found between Matrix SRT values in the quiet and Matrix SRT values in noise. Similarly, the correlation between PTA values and intelligibility scores in noise was also not significant. Therefore, it may not be possible to assess the intelligibility performance of CI users using test batteries performed in quiet conditions. The Matrix Test can be used to assess the benefit of CI users from their systems in everyday life, since it is possible to perform intelligibility test with the Matrix test using a material that CI users experience in their everyday life and it is possible to assess their difficulty in speech discrimination in noisy conditions they have to cope with.

Regulating for responsibility: reputation and social media

The framework brought forward by the United Kingdom's Defamation Act 2013 underlines a traditional hierarchy of expression in which news media are viewed as high-level speech. Although of a different form, social media are a dominant means of expression. The current study explores the rationale for a more robust and forceful discussion of responsibility in speech on social media platforms. The underlying premise here is that speech should be viewed as a qualified good and that a more appropriate paradigm is one found in the phrase ‘freedom to participate’.

Harmonic and intermodulation performance of hearing aids under large sound pressure levels

A mathematical model for the input–output functions of the hearing aids is presented. The model, basically a sine-series function, can easily yield closed-form expressions for the amplitudes of the harmonic and intermodulation components of the output when the input is formed of multitone large sound pressure levels. The special case of two-tone equal-amplitude incident pressure level is considered in detail. The results show that the third-order intermodulation component is always higher than the third-harmonic component. The results also show that the input–output functions exhibiting peak clipping exhibit the worst harmonic and intermodulation performance for large sound pressure levels. The simple wide-dynamic range compression (WDRC) input–output function exhibits the best harmonic and intermodulation distortion performance under large sound pressure levels. These results are consistent with the previously reported observations and reveal that there is a correlation between intelligibility of high-level speech and the harmonic and intermodulation performance of the hearing aid under large sound pressure levels.

Effects of Audibility and Multichannel Wide Dynamic Range Compression on Consonant Recognition for Listeners with Severe Hearing Loss

This study examined the effects of multichannel wide-dynamic range compression (WDRC) amplification and stimulus audibility on consonant recognition and error patterns. Listeners had either severe or mild to moderate sensorineural hearing loss. Each listener was monaurally fit with a wearable hearing aid using typical clinical procedures, frequency-gain parameters, and a hybrid of clinically prescribed compression ratios for desired sensation level () and NAL-NL (). Consonant-vowel nonsense syllables were presented in soundfield at multiple input levels (50, 65, 80 dB SPL). Test conditions were four-channel fast-acting WDRC amplification and a control compression limiting (CL) amplification condition. Listeners identified the stimulus heard from choices presented on an on-screen display. A between-subject repeated measures design was used to evaluate consonant recognition and consonant confusion patterns. Fast-acting WDRC provided a considerable audibility advantage at 50 dB SPL, especially for listeners with severe hearing loss. Listeners with mild to moderate hearing loss received less audibility improvement from the fast-acting WDRC amplification, for conversational and high level speech, when compared with listeners with severe hearing loss. Analysis of WDRC benefit scores revealed that listeners had slightly lower scores with fast-acting WDRC amplification (relative to CL) when WDRC provided minimal improvement in audibility. The negative effect was greater for listeners with mild to moderate hearing loss compared with their counterparts with severe hearing loss. All listeners, but particularly the severe loss group, benefited from fast-acting WDRC amplification for low-level speech. For conversational and higher speech levels (i.e., when WDRC does not confer a significant audibility advantage), fast-acting WDRC amplification seems to slightly degrade performance. Listeners' consonant confusion patterns suggest that this negative effect may be partly due to fast-acting WDRC-induced distortions, which alter specific consonant features. In support of this view, audibility accounted for a greater percentage of the variance in listeners' performance with CL amplification compared with fast-acting WDRC amplification.

Ranking Hearing Aid Input–Output Functions for Understanding Low-, Conversational-, and High-Level Speech in Multitalker Babble

To determine the rankings of 6 input-output functions for understanding low-level, conversational, and high-level speech in multitalker babble without manipulating volume control for listeners with normal hearing, flat sensorineural hearing loss, and mildly sloping sensorineural hearing loss. Peak clipping, compression limiting, and 4 wide dynamic range compression (WDRC) input-output functions were compared in a repeated-measure design. Interactions among the compression characteristics were minimized. Speech and babble were processed and recorded at 3 input levels: 45, 65, and 90 dB sound pressure level. Speech recognition of 3 groups of listeners (n = 6/group) was tested for speech processed by each input-output function and at each input level. Input-output functions that made low-level speech audible and high-level speech less distorted by avoiding peak clipping or severe compression yielded higher speech recognition scores. These results are consistent with previous findings in the literature. WDRCs with the low compression ratio region extended to a high input level or with a high compression limiting threshold were the best for speech recognition in babble when the hearing aid user cannot or does not want to manipulate the volume control. Future studies on subjective preferences of different input-output functions are needed.

Temporally Jittered Speech Produces Performance Intensity, Phonetically Balanced Rollover in Young Normal-Hearing Listeners

This study investigates whether temporally jittered stimuli will produce performance-intensity, phonetically balanced (PI-PB) rollover in young adults with normal hearing. Although not yet explicitly stated in the literature, there is clinical and theoretical evidence to suggest that PI-PB rollover, such as that found in cases of acoustic neuroma, is caused by neural dyssynchrony in the auditory system. Sixteen participants were tested with intact and temporally jittered word lists in quiet at 40, 55, 65, and uncomfortable listening level -5 dB HL. The results show significant rollover in the jittered but not the intact conditions. The results are consistent with the existing evidence that suggests that neural PI-PB rollover is caused by decreased neural synchrony and support the claim that temporal jitter simulates neural dyssynchrony. Furthermore, these results are consistent with the hypothesis that synchrony coding plays an important role in the perception of high-level speech.

A layered neural network interfaced with a cochlear model for the study of speech encoding in the auditory system

Our recent development of a computational cochlear-nucleus-like network model for the study of speech encoding mechanisms associated with parts of the auditory system central to the auditory nerve is described in this paper. This network model is based on physiological grounds, and is designed to gracefully interface with a cochlear model established earlier that incorporates a biophysically motivated dynamic nonlinearity in the basilar-membrane filtering function. This study addresses a longstanding issue in auditory research; that is, how the unique spatial–temporal patterns, shaped by the cochlea's nonlinear filtering, in the auditory nerve data in response to high-level speech sounds can be transformed into a rate-place code in the auditory system in a physiologically plausible manner? Detailed neural mechanisms implemented in the model include neural inhibition, coincidence detection, short-term temporal integration of post synaptic potentials for action potential generation, and a conjectural temporal-to-rate conversion mechanism requiring the membrane time constant of a neuron to monotonically decrease with the neuron's CF. Model simulation experiments using both synthetic and natural speech utterances demonstrate that the auditory rate-place code constructed at the output of the network model is capable of reliable representation, with possible modification and/or enhancement, of the prominent spectral characteristics of the utterances displayed in wideband spectrograms. The effectiveness of the rate-place code is demonstrated, with examples taken from TIMIT corpus, to be universal across all classes of speech sounds including vowels, liquids, fricatives, nasals, and stops.

Using talking computers to remediate reading and spelling disabilities: The critical role of the print-to-sound unit

Talking computers employing computer-generated speech feedback have been used to remediate the literacy skills of dyslexic readers. A computer program is described that employs DECtalk, a highlevel speech synthesizer, to narrate instruction involving intensive training in identifying whole words or in identifying and blending word segments corresponding to onsets, rimes, and phonemes. Procedures for developing individualized instruction are described as well as for constructing and editing the speech and graphics features of the program. Neurologically impaired dyslexic children trained with this program achieved greater acquisition and transfer of word recognition skill when their training involved segmented rather than whole word feedback.

Extending Database Support for Coordination Among Agents

Coordination and collaboration are naturally used by groups for carrying out activities and solving problems that require cooperation. However, getting a set of computer agents to do the same has been a problem – primarily addressed by the AI community and recently by the database community as workflow and process management problems (for example, in business processes, electronic commerce, logistics). Not surprisingly, the problem has been addressed at different levels of abstraction by the two communities. Coordination protocols as well as task and result sharing have been investigated by the AI community; specification of alternative transaction models to meet the requirements of non-traditional applications, and their execution have been addressed by the database community. It is evident that there is a need for bringing the two approaches together to develop systems that support cooperative problem solving. This paper – argues for the use of active databases in general and active capability in particular as an enabling technology for cooperative problem solving and cooperative information systems – details a novel approach for supporting task sharing, a key aspect of CPS, using active capability – elaborates on a methodology for mapping task shared protocols expressed in high level speech acts to Event Condition-Action (ECA) rules.

Short-term memory impairment and spontaneous speech

The spontaneous speech of a patient with a specific repetition deficit was studied. The amount of pausing and of paraphasic errors was found to be normal. It was concluded that the repetition difficulty arose from an auditory-verbal short-term memory impairment and not a high-level speech production deficit. It was further concluded that the STM store involved was not part of the speech output system.