Time-reversed Speech Research Articles

The Concise Language Paradigm (CLaP), a framework for studying the intersection of comprehension and production: electrophysiological properties.

Studies investigating language commonly isolate one modality or process, focusing on comprehension or production. Here, we present a framework for a paradigm that combines both: the Concise Language Paradigm (CLaP), tapping into comprehension and production within one trial. The trial structure is identical across conditions, presenting a sentence followed by a picture to be named. We tested 21 healthy speakers with EEG to examine three time periods during a trial (sentence, pre-picture interval, picture onset), yielding contrasts of sentence comprehension, contextually and visually guided word retrieval, object recognition, and naming. In the CLaP, sentences are presented auditorily (constrained, unconstrained, reversed), and pictures appear as normal (constrained, unconstrained, bare) or scrambled objects. Imaging results revealed different evoked responses after sentence onset for normal and time-reversed speech. Further, we replicated the context effect of alpha-beta power decreases before picture onset for constrained relative to unconstrained sentences, and could clarify that this effect arises from power decreases following constrained sentences. Brain responses locked to picture-onset differed as a function of sentence context and picture type (normal vs. scrambled), and naming times were fastest for pictures in constrained sentences, followed by scrambled picture naming, and equally fast for bare and unconstrained picture naming. Finally, we also discuss the potential of the CLaP to be adapted to different focuses, using different versions of the linguistic content and tasks, in combination with electrophysiology or other imaging methods. These first results of the CLaP indicate that this paradigm offers a promising framework to investigate the language system.

Listening Preference for Child-Directed Speech Versus Time-Reversed Speech in Moderate-Preterm Infants Compared to Full-Term Infants.

The purpose of the present study was to examine whether there is a listening preference for child-directed speech (CDS) over backward speech in moderate-preterm infants (MPIs). Eighteen MPIs of gestational age of 32.0 weeks (range: 32-34.06 weeks), chronological age of 8.09 months, and maturation age of 6.48 months served as the experimental group. The two control groups consisted of a total of 36 infants-20 full-term infants matched for chronological age and 16 full-term infants matched for maturation age. The infants were tested using the central fixation procedure and were presented with 16 trials of CDS and backward speech. A follow-up was conducted 5 years after the initial experiment using a developmental and a five-item parent questionnaire. MPIs did not demonstrate a preference for CDS over backward speech, whereas both control groups demonstrated a listening preference for CDS over backward speech. MPIs showed a delayed use of first words and word combinations and lower scores on the five-item questionnaire compared to term infants. Twelve MPIs (67%) did not demonstrate a preference for CDS over backward speech. Four of them (33%) were later diagnosed with neurodevelopmental disorders. The lack of preference for CDS over backward speech in the MPIs group suggests delayed developmental pattern of speech processing compared to full-term peers. Delays in neurological maturation as well as listening experience in an unregulated environment outside the uterus during a sensitive period of brain development may affect the recognition of phonological and prosodic patterns that support listening preference for speech over backward speech.

Distinct roles of delta- and theta-band neural tracking for sharpening and predictive coding of multi-level speech features during spoken language processing.

The brain tracks and encodes multi-level speech features during spoken language processing. It is evident that this speech tracking is dominant at low frequencies (<8 Hz) including delta and theta bands. Recent research has demonstrated distinctions between delta- and theta-band tracking but has not elucidated how they differentially encode speech across linguistic levels. Here, we hypothesised that delta-band tracking encodes prediction errors (enhanced processing of unexpected features) while theta-band tracking encodes neural sharpening (enhanced processing of expected features) when people perceive speech with different linguistic contents. EEG responses were recorded when normal-hearing participants attended to continuous auditory stimuli that contained different phonological/morphological and semantic contents: (1) real-words, (2) pseudo-words and (3) time-reversed speech. We employed multivariate temporal response functions to measure EEG reconstruction accuracies in response to acoustic (spectrogram), phonetic and phonemic features with the partialling procedure that singles out unique contributions of individual features. We found higher delta-band accuracies for pseudo-words than real-words and time-reversed speech, especially during encoding of phonetic features. Notably, individual time-lag analyses showed that significantly higher accuracies for pseudo-words than real-words started at early processing stages for phonetic encoding (<100 ms post-feature) and later stages for acoustic and phonemic encoding (>200 and 400 ms post-feature, respectively). Theta-band accuracies, on the other hand, were higher when stimuli had richer linguistic content (real-words > pseudo-words > time-reversed speech). Such effects also started at early stages (<100 ms post-feature) during encoding of all individual features or when all features were combined. We argue these results indicate that delta-band tracking may play a role in predictive coding leading to greater tracking of pseudo-words due to the presence of unexpected/unpredicted semantic information, while theta-band tracking encodes sharpened signals caused by more expected phonological/morphological and semantic contents. Early presence of these effects reflects rapid computations of sharpening and prediction errors. Moreover, by measuring changes in EEG alpha power, we did not find evidence that the observed effects can be solitarily explained by attentional demands or listening efforts. Finally, we used directed information analyses to illustrate feedforward and feedback information transfers between prediction errors and sharpening across linguistic levels, showcasing how our results fit with the hierarchical Predictive Coding framework. Together, we suggest the distinct roles of delta and theta neural tracking for sharpening and predictive coding of multi-level speech features during spoken language processing.

Study on neural entrainment to continuous speech using dynamic source connectivity analysis

Objective. Many recent studies investigating the processing of continuous natural speech have employed electroencephalography (EEG) due to its high temporal resolution. However, most of these studies explored the response mechanism limited to the electrode space. In this study, we intend to explore the underlying neural processing in the source space, particularly the dynamic functional interactions among different regions during neural entrainment to speech. Approach. We collected 128-channel EEG data while 22 participants listened to story speech and time-reversed speech using a naturalistic paradigm. We compared three different strategies to determine the best method to estimate the neural tracking responses from the sensor space to the brain source space. After that, we used dynamic graph theory to investigate the source connectivity dynamics among regions that were involved in speech tracking. Main result. By comparing the correlations between the predicted neural response and the original common neural response under the two experimental conditions, we found that estimating the common neural response of participants in the electrode space followed by source localization of neural responses achieved the best performance. Analysis of the distribution of brain sources entrained to story speech envelopes showed that not only auditory regions but also frontoparietal cognitive regions were recruited, indicating a hierarchical processing mechanism of speech. Further analysis of inter-region interactions based on dynamic graph theory found that neural entrainment to speech operates across multiple brain regions along the hierarchical structure, among which the bilateral insula, temporal lobe, and inferior frontal gyrus are key brain regions that control information transmission. All of these information flows result in dynamic fluctuations in functional connection strength and network topology over time, reflecting both bottom-up and top-down processing while orchestrating computations toward understanding. Significance. Our findings have important implications for understanding the neural mechanisms of the brain during processing natural speech stimuli.

Do individual differences in lexical reliance reflect states or traits?

Research suggests that individuals differ in the degree to which they rely on lexical information to support speech perception. However, the locus of these differences is not yet known; nor is it known whether these individual differences reflect a context-dependent “state” or a stable listener “trait.” Here we test the hypothesis that individual differences in lexical reliance are a stable trait that is linked to individuals' relative weighting of lexical and acoustic-phonetic information for speech perception. At each of two sessions, listeners (n = 73) completed a Ganong task, a phonemic restoration task, and a locally time-reversed speech task – three tasks that have been used to demonstrate a lexical influence on speech perception. Robust lexical effects on speech perception were observed for each task in the aggregate. Individual differences in lexical reliance were stable across sessions; however, relationships among the three tasks in each session were weak. For the Ganong and locally time-reversed speech tasks, increased reliance on lexical information was associated with weaker reliance on acoustic-phonetic information. Collectively, these results (1) provide some evidence to suggest that individual differences in lexical reliance for a given task are a stable reflection of the relative weighting of acoustic-phonetic and lexical cues for speech perception in that task, and (2) highlight the need for a better understanding of the psychometric characteristics of tasks used in the psycholinguistic domain to build theories that can accommodate individual differences in mapping speech to meaning.

Loudness and rate perception in Parkinson's disease

In speech communication, Parkinson’s disease (PD) is typically discussed in terms of its production consequences. However, a small but growing literature indicates that PD may also result in deficits in the perception of loudness and rate information in speech. In the present study, we focus on the extent to which the perceptual deficits in PD are limited to speech or shared with other modalities, and the degree to which these deficits are dependent solely on perception or are also found in tasks of learning. To assess these questions, people with PD and age-matched controls perform two tasks: an AXB task of speech rate or intensity discrimination, and a simple learning task testing participants’ ability to group versions of a sentence based on their rate or intensity properties. For the AXB task, participants perform three additional versions of the task with stimuli that vary in their speech-like properties: time-reversed speech, pure tones, or visualizations. Preliminary data indicates that for control participants, all the continua are discriminated at above-chance performance and that participants can learn to group sentences based on both rate and intensity. Performance for the people with Parkinson’s Disease displays a great deal of variability.

The time-course of linguistic interference during native and non-native speech-in-speech listening

Recognizing speech in a noisy background is harder when the background is time-forward than for time-reversed speech, a masker direction effect, and harder when the masker is in a known rather than an unknown language, indicating linguistic interference. We examined the masker direction effect when the masker was a known vs unknown language and calculated performance over 50 trials to assess differential masker adaptation. In experiment 1, native English listeners transcribing English sentences showed a larger masker direction effect with English than Mandarin maskers. In experiment 2, Mandarin non-native speakers of English transcribing Mandarin sentences showed a mirror pattern. Both experiments thus support the target-masker linguistic similarity hypothesis, where interference is maximal when target and masker languages are the same. In experiment 3, Mandarin non-native speakers of English transcribing English sentences showed comparable results for English and Mandarin maskers. Non-native listening is therefore consistent with the known-language interference hypothesis, where interference is maximal when the masker language is known to the listener, whether or not it matches the target language. A trial-by-trial analysis showed that the masker direction effect increased over time during native listening but not during non-native listening. The results indicate different target-to-masker streaming strategies during native and non-native speech-in-speech listening.

Detection of Brain Network Communities During Natural Speech Comprehension From Functionally Aligned EEG Sources.

In recent years, electroencephalograph (EEG) studies on speech comprehension have been extended from a controlled paradigm to a natural paradigm. Under the hypothesis that the brain can be approximated as a linear time-invariant system, the neural response to natural speech has been investigated extensively using temporal response functions (TRFs). However, most studies have modeled TRFs in the electrode space, which is a mixture of brain sources and thus cannot fully reveal the functional mechanism underlying speech comprehension. In this paper, we propose methods for investigating the brain networks of natural speech comprehension using TRFs on the basis of EEG source reconstruction. We first propose a functional hyper-alignment method with an additive average method to reduce EEG noise. Then, we reconstruct neural sources within the brain based on the EEG signals to estimate TRFs from speech stimuli to source areas, and then investigate the brain networks in the neural source space on the basis of the community detection method. To evaluate TRF-based brain networks, EEG data were recorded in story listening tasks with normal speech and time-reversed speech. To obtain reliable structures of brain networks, we detected TRF-based communities from multiple scales. As a result, the proposed functional hyper-alignment method could effectively reduce the noise caused by individual settings in an EEG experiment and thus improve the accuracy of source reconstruction. The detected brain networks for normal speech comprehension were clearly distinctive from those for non-semantically driven (time-reversed speech) audio processing. Our result indicates that the proposed source TRFs can reflect the cognitive processing of spoken language and that the multi-scale community detection method is powerful for investigating brain networks.

Developmental changes in the brain response to speech during the first year of life: A near-infrared spectroscopy study of dutch-learning infants

Language acquisition in the first year of life plays an important role in human development. Although recent research has increased our knowledge of early language development, the origins and developmental trajectories of language processing during infancy are still being debated. One important issue is whether the infant brain has already developed adult-like functional cortical specialization and lateralization for speech and language processing. Although a relatively large number of previous studies have investigated the cortical specialization for speech processing in newborns by comparing responses to normal speech with responses to time-reversed speech as a control, the subsequent development of this differential response is less well understood. In the current study, our primary goal was, therefore, to expand our knowledge of functional specialization for speech during the first year of life. By using functional near-infrared spectroscopy (fNIRS), we examined the hemodynamic responses in the temporal cortex to forward-going speech in the native language, Dutch, and backward Dutch in seventy-eight infants aged 5 and 10 months. We observed that five-month-old infants show bilateral activation to both forward and backward speech without obvious hemispheric lateralization for language, while ten-month-old infants show greater left-lateralized inverted responses (e.g., reduction in oxyhaemoglobin and increase in deoxyhaemoglobin) to forward speech than to backward speech. Overall, our findings indicate that the neural correlates of language processing undergo developmental changes in the first year of life.

Erratum: Intelligibility of chimeric locally time-reversed speech: Relative contribution of four frequency bands [ JASA Express Lett. 1(6), 065201 (2021)

Erratum: Intelligibility of chimeric locally time-reversed speech: Relative contribution of four frequency bands [ JASA Express Lett. <b>1</b>(6), 065201 (2021)

Orienting Auditory Attention through Vision: theImpactofMonaural Listening.

The understanding of linguistic messages can be made extremely complex by the simultaneous presence of interfering sounds, especially when they are also linguistic in nature. In two experiments, we tested if visual cues directing attention to spatial or temporal components of speech in noise can improve its identification. The hearing-in-noise task required identification of a five-digit sequence (target) embedded in a stream of time-reversed speech. Using a custom-built device located in front of the participant, we delivered visual cues to orient attention to the location of target sounds and/or their temporal window. In Exp. 1 ( n = 14), we validated this visual-to-auditory cueing method in normal-hearing listeners, tested under typical binaural listening conditions. In Exp. 2 ( n = 13), we assessed the efficacy of the same visual cues in normal-hearing listeners wearing a monaural ear plug, to study the effects of simulated monaural and conductive hearing loss on visual-to-auditory attention orienting. While Exp. 1 revealed a benefit of both spatial and temporal visual cues for hearing in noise, Exp. 2 showed that only the temporal visual cues remained effective during monaural listening. These findings indicate that when the acoustic experience is altered, visual-to-auditory attention orienting is more robust for temporal compared to spatial attributes of the auditory stimuli. These findings have implications for the relation between spatial and temporal attributes of sound objects, and when planning devices to orient audiovisual attention for subjects suffering from hearing loss.

Infants' Preference for Child-Directed Speech Over Time-Reversed Speech in On-Channel and Off-Channel Masking.

Purpose This study aims to examine the development of auditory selective attention to speech in noise by examining the ability of infants to prefer child-directed speech (CDS) over time-reversed speech (TRS) presented in "on-channel" and "off-channel" noise. Method A total of 32 infants participated in the study. Sixteen typically developing infants were tested at 7 and 11 months of age using the central fixation procedure with CDS and TRS in two types of noise at +10 dB signal-to-noise ratio. One type of noise was an "on-channel" masker with a spectrum overlapping that of the CDS (energetic masking), and the second was an "off-channel" masker with frequencies that were outside the spectrum of the CDS (distractive masking). An additional group of sixteen 11-month-old infants were tested in quiet and served as controls for the "off-frequency" masker condition. Results Infants preferred CDS over TRS in both age groups, but this preference was more pronounced with "off-channel" masker regardless of age. Also, older infants demonstrated longer looking time for the target stimuli when presented with an "off-channel" masker compared to the "on-channel" masker. Looking time in quiet was similar to looking time in the "off-channel" condition, and looking time for CDS was longer in quiet compared to the "on-channel" condition. Conclusions These findings support the notion that (a) infants as young as 7 months of age are already showing preference for speech in noise, regardless of type of masker; (b) by 11 months of age, listening with the "off-channel" condition did not yield different results than in quiet. Thus, by 11 months of age, infants' cognitive-attentional abilities may be more developed.

Intelligibility of chimeric locally time-reversed speech: Relative contribution of four frequency bands.

Intelligibility of four-band speech stimuli was investigated (n = 18), such that only one of the frequency bands was preserved, whereas other bands were locally time-reversed (segment duration: 75-300 ms), or vice versa. Intelligibility was best retained (82% at 75 ms) when the second lowest band (540-1700 Hz) was preserved. When the same band was degraded, the largest drop (10% at 300 ms) occurred. The lowest and second highest bands contributed similarly less strongly to intelligibility. The highest frequency band contributed least. A close connection between the second lowest frequency band and sonority was suggested.

Perceptual restoration of locally time-reversed speech: Non-native listeners\u2019 performance in their L2 vs. L1

Nonnative listeners are generally not as good as native listeners in perceptually restoring degraded speech and understand what was being said. The current study investigates how nonnative listeners of English (namely, native Japanese speakers who learned English as a second language) perceptually restore temporally distorted speech in their L2 English as compared with native English listeners (L1 English) reported in Ishida et al. (Cognition, 151, 68–75, 2016), and as compared with the listeners’ native tongue (L1 Japanese). In the experiment, listeners listened to locally time-reversed words and pseudowords in their L2 English and L1 Japanese where every 10, 30, 50, 70, 90, or 110 ms of speech signal was flipped in time—these stimuli contained either many fricatives or stops. The results suggested that the intelligibility of locally time-reversed words and pseudowords deteriorated as the length of reversed segments increased in both listeners’ L2 English and L1 Japanese, while listeners understood locally time-reversed speech more in their L1 Japanese. In addition, lexical context supported perceptual restoration in both listeners’ L1 Japanese and L2 English, while phonemic constituents affected perceptual restoration significantly only in listeners’ L1. On the other hand, locally time-reversed words and pseudowords in L1 Japanese were much more intelligible than those in L1 English reported in Ishida et al. It is possible that the intelligibility of temporally distorted lexical items depends on the structure of basic linguistic units in each language, and the Japanese language might have a unique characteristic because of its CV and V structure.

Phonemic restoration of interrupted locally time-reversed speech

Intelligibility of temporally degraded speech was investigated with locally time-reversed speech (LTR) and its interrupted version (ILTR). Control stimuli comprising interrupted speech (I) were also included. Speech stimuli consisted of 200 Japanese meaningful sentences. In interrupted stimuli, speech segments were alternated with either silent gaps or pink noise bursts. The noise bursts had a level of − 10, 0 or + 10 dB relative to the speech level. Segment duration varied from 20 to 160 ms for ILTR sentences, but was fixed at 160 ms for I sentences. At segment durations between 40 and 80 ms, severe reductions in intelligibility were observed for ILTR sentences, compared with LTR sentences. A substantial improvement in intelligibility (30–33%) was observed when 40-ms silent gaps in ILTR were replaced with 0- and + 10-dB noise. Noise with a level of − 10 dB had no effect on the intelligibility. These findings show that the combined effects of interruptions and temporal reversal of speech segments on intelligibility are greater than the sum of each individual effect. The results also support the idea that illusory continuity induced by high-level noise bursts improves the intelligibility of ILTR and I sentences.

Birdsong fails to support object categorization in human infants.

Recent evidence reveals a precocious link between language and cognition in human infants: listening to their native language supports infants’ core cognitive processes, including object categorization, and does so in a way that other acoustic signals (e.g., time-reversed speech; sine-wave tone sequences) do not. Moreover, language is not the only signal that confers this cognitive advantage: listening to vocalizations of non-human primates also supports object categorization in 3- and 4-month-olds. Here, we move beyond primate vocalizations to clarify the breadth of acoustic signals that promote infant cognition. We ask whether listening to birdsong, another naturally produced animal vocalization, also supports object categorization in 3- and 4-month-old infants. We report that listening to zebra finch song failed to confer a cognitive advantage. This outcome brings us closer to identifying a boundary condition on the range of non-linguistic acoustic signals that initially support infant cognition.

Cortical Tracking of Speech in Delta Band Relates to Individual Differences in Speech in Noise Comprehension in Older Adults.

Understanding speech in adverse listening environments is challenging for older adults. Individual differences in pure tone averages and working memory are known to be critical indicators of speech in noise comprehension. Recent studies have suggested that tracking of the speech envelope in cortical oscillations <8 Hz may be an important mechanism related to speech comprehension by segmenting speech into words and phrases (delta, 1 to 4 Hz) or phonemes and syllables (theta, 4 to 8 Hz). The purpose of this study was to investigate the extent to which individual differences in pure tone averages, working memory, and cortical tracking of the speech envelope relate to speech in noise comprehension in older adults. Cortical tracking of continuous speech was assessed using electroencephalography in older adults (60 to 80 years). Participants listened to speech in quiet and in the presence of noise (time-reversed speech) and answered comprehension questions. Participants completed Forward Digit Span and Backward Digit Span as measures of working memory, and pure tone averages were collected. An index of reduction in noise (RIN) was calculated by normalizing the difference between raw cortical tracking in quiet and in noise. Comprehension question performance was greater for speech in quiet than for speech in noise. The relationship between RIN and speech in noise comprehension was assessed while controlling for the effects of individual differences in pure tone averages and working memory. Delta band RIN correlated with speech in noise comprehension, while theta band RIN did not. Cortical tracking by delta oscillations is robust to the effects of noise. These findings demonstrate that the magnitude of delta band RIN relates to individual differences in speech in noise comprehension in older adults. Delta band RIN may serve as a neural metric of speech in noise comprehension beyond the effects of pure tone averages and working memory.

Intelligibility of chimeric locally time-reversed speech.

The intelligibility of chimeric locally time-reversed speech was investigated. Both (1) the boundary frequency between the temporally degraded band and the non-degraded band and (2) the segment duration were varied. Japanese mora accuracy decreased if the width of the degraded band or the segment duration increased. Nevertheless, the chimeric stimuli were more intelligible than the locally time-reversed controls. The results imply that the auditory system can use both temporally degraded speech information and undamaged speech information over different frequency regions in the processing of the speech signal, if the amplitude envelope in the frequency range of 840-1600 Hz was preserved.

Hearing Aids Do Not Alter Cortical Entrainment to Speech at Audible Levels in Mild-to-Moderately Hearing-Impaired Subjects.

BackgroundCortical entrainment to speech correlates with speech intelligibility and attention to a speech stream in noisy environments. However, there is a lack of data on whether cortical entrainment can help in evaluating hearing aid fittings for subjects with mild to moderate hearing loss. One particular problem that may arise is that hearing aids may alter the speech stimulus during (pre-)processing steps, which might alter cortical entrainment to the speech. Here, the effect of hearing aid processing on cortical entrainment to running speech in hearing impaired subjects was investigated.MethodologySeventeen native English-speaking subjects with mild-to-moderate hearing loss participated in the study. Hearing function and hearing aid fitting were evaluated using standard clinical procedures. Participants then listened to a 25-min audiobook under aided and unaided conditions at 70 dBA sound pressure level (SPL) in quiet conditions. EEG data were collected using a 32-channel system. Cortical entrainment to speech was evaluated using decoders reconstructing the speech envelope from the EEG data. Null decoders, obtained from EEG and the time-reversed speech envelope, were used to assess the chance level reconstructions. Entrainment in the delta- (1–4 Hz) and theta- (4–8 Hz) band, as well as wideband (1–20 Hz) EEG data was investigated.ResultsSignificant cortical responses could be detected for all but one subject in all three frequency bands under both aided and unaided conditions. However, no significant differences could be found between the two conditions in the number of responses detected, nor in the strength of cortical entrainment. The results show that the relatively small change in speech input provided by the hearing aid was not sufficient to elicit a detectable change in cortical entrainment.ConclusionFor subjects with mild to moderate hearing loss, cortical entrainment to speech in quiet at an audible level is not affected by hearing aids. These results clear the pathway for exploring the potential to use cortical entrainment to running speech for evaluating hearing aid fitting at lower speech intensities (which could be inaudible when unaided), or using speech in noise conditions.

Distinct Patterns of Functional Connectivity During the Comprehension of Natural, Narrative Speech.

Recent continuous task studies, such as narrative speech comprehension, show that fluctuations in brain functional connectivity (FC) are altered and enhanced compared to the resting state. Here, we characterized the fluctuations in FC during comprehension of speech and time-reversed speech conditions. The correlations of Hilbert envelope of source-level EEG data were used to quantify FC between spatially separate brain regions. A symmetric multivariate leakage correction was applied to address the signal leakage issue before calculating FC. The dynamic FC was estimated based on a sliding time window. Then, principal component analysis (PCA) was performed on individually concatenated and temporally concatenated FC matrices to identify FC patterns. We observed that the mode of FC induced by speech comprehension can be characterized with a single principal component. The condition-specific FC demonstrated decreased correlations between frontal and parietal brain regions and increased correlations between frontal and temporal brain regions. The fluctuations of the condition-specific FC characterized by a shorter time demonstrated that dynamic FC also exhibited condition specificity over time. The FC is dynamically reorganized and FC dynamic pattern varies along a single mode of variation during speech comprehension. The proposed analysis framework seems valuable for studying the reorganization of brain networks during continuous task experiments.