Rhythmic Expectation Research Articles

Functions and specificity of metaphorical complexes in D. Kehlmann’s novel «Tyll»

The article is devoted to the analysis of the linguistic representation and functional features of the metaphorical complexes death and movement in the German-language novel discourse. The research was based on the novel «Tyll» (2017) written by the modern German writer D. Kehlmann. The novel tells about the character of German medieval legends Till Eulenspiegel portrayed as vagabond and artist, placed in the landscape of the Thirty Years’ War. The relevance of the research is due to the lack of systematic studies of the novel metaphor and its dominant role in the process of meaning generation in literary texts at the «new turn of the century». The author used an integral approach to the problem under study. The initial theoretical premise of the study is the recognition of the serious cognitive potential of metaphor, realized in the process of artistic cognition. As a result of the research, it was found that the metaphorical complex death is embodied by D. Kehlmann in the novel by sequence of figurative rows that create a metaphorical connection between the arrival of the harbinger of death Till Eulenspiegel and the onset of War. The author suggests considering the process of formation of the metaphorical complex movement in a series of text fragments created with the participation of verbs with semantics of movement, spatial prepositions and adverbs of repetition. Attempting to characterize the rhythmic specificity of the novel text under study, the author comes to the conclusion that the main method used by D. Kehlmann is justification of rhythmic expectation. The rhythm of the text is consistent with the conceptual dance of the main character through subjectively ranged gradation rows with an ascending increment of semantic and emotionally expressive significance, which determine the author’s modality. Metaphorical complexes death and movement have prognostic and compositional functions. The results obtained confirm the metaphoricity of the novel under study and the necessity to analyze other metaphorical complexes as part of the central novel metaphor as an important category of modern German literature.

Violation of rhythmic expectancies can elicit late frontal gamma activity nested in theta oscillations.

Rhythm processing involves building expectations according to the hierarchical temporal structure of auditory events. Although rhythm processing has been addressed in the context of predictive coding, the properties of the oscillatory response in different cortical areas are still not clear. We explored the oscillatory properties of the neural response to rhythmic incongruence and the cross‐frequency coupling between multiple frequencies to further investigate the mechanisms underlying rhythm perception. We designed an experiment to investigate the neural response to rhythmic deviations in which the tone either arrived earlier than expected or the tone in the same metrical position was omitted. These two manipulations modulate the rhythmic structure differently, with the former creating a larger violation of the general structure of the musical stimulus than the latter. Both deviations resulted in an MMN response, whereas only the rhythmic deviant resulted in a subsequent P3a. Rhythmic deviants due to the early occurrence of a tone, but not omission deviants, seemed to elicit a late high gamma response (60–80 Hz) at the end of the P3a over the left frontal region, which, interestingly, correlated with the P3a amplitude over the same region and was also nested in theta oscillations. The timing of the elicited high‐frequency gamma oscillations related to rhythmic deviation suggests that it might be related to the update of the predictive neural model, corresponding to the temporal structure of the events in higher‐level cortical areas.

From beat tracking to beat expectation: Cognitive-based beat tracking for capturing pulse clarity through time.

Pulse is the base timing to which western music is commonly notated, generally expressed by a listener by performing periodic taps with their hand or foot. This cognitive construction helps organize the perception of timed events in music and is the most basic expectation in rhythms. The analysis of expectations, and more specifically the strength with which the beat is felt—the pulse clarity—has been used to analyze affect in music. Most computational models of pulse clarity, and rhythmic expectation in general, analyze the input as a whole, without exhibiting changes through a rhythmic passage. We present Tactus Hypothesis Tracker (THT), a model of pulse clarity over time intended for symbolic rhythmic stimuli. The model was developed based on ideas of beat tracking models that extract beat times from musical stimuli. Our model also produces possible beat interpretations for the rhythm, a fitness score for each interpretation and how these evolve in time. We evaluated the model’s pulse clarity by contrasting against tapping variability of human annotators achieving results comparable to a state-of-the-art pulse clarity model. We also analyzed the clarity metric dynamics on synthetic data that introduced changes in the beat, showing that our model presented doubt in the pulse estimation process and adapted accordingly to beat changes. Finally, we assessed if the beat tracking generated by the model was correct regarding listeners tapping data. We compared our beat tracking results with previous beat tracking models. The THT model beat tracking output showed generally correct estimations in phase but exhibits a bias towards a musically correct subdivision of the beat.

Rhythmic Temporal Expectation Boosts Neural Activity by Increasing Neural Gain.

Temporal orienting improves sensory processing, akin to other top-down biases. However, it is unknown whether these improvements reflect increased neural gain to any stimuli presented at expected time points, or specific tuning to task-relevant stimulus aspects. Furthermore, while other top-down biases are selective, the extent of trade-offs across time is less well characterized. Here, we tested whether gain and/or tuning of auditory frequency processing in humans is modulated by rhythmic temporal expectations, and whether these modulations are specific to time points relevant for task performance. Healthy participants (N = 23) of either sex performed an auditory discrimination task while their brain activity was measured using magnetoencephalography/electroencephalography (M/EEG). Acoustic stimulation consisted of sequences of brief distractors interspersed with targets, presented in a rhythmic or jittered way. Target rhythmicity not only improved behavioral discrimination accuracy and M/EEG-based decoding of targets, but also of irrelevant distractors preceding these targets. To explain this finding in terms of increased sensitivity and/or sharpened tuning to auditory frequency, we estimated tuning curves based on M/EEG decoding results, with separate parameters describing gain and sharpness. The effect of rhythmic expectation on distractor decoding was linked to gain increase only, suggesting increased neural sensitivity to any stimuli presented at relevant time points.SIGNIFICANCE STATEMENT Being able to predict when an event may happen can improve perception and action related to this event, likely due to the alignment of neural activity to the temporal structure of stimulus streams. However, it is unclear whether rhythmic increases in neural sensitivity are specific to task-relevant targets, and whether they competitively impair stimulus processing at unexpected time points. By combining magnetoencephalography and encephalographic recordings, neural decoding of auditory stimulus features, and modeling, we found that rhythmic expectation improved neural decoding of both relevant targets and irrelevant distractors presented and expected time points, but did not competitively impair stimulus processing at unexpected time points. Using a quantitative model, these results were linked to nonspecific neural gain increases due to rhythmic expectation.

The Temporal Prediction of Stress in Speech and Its Relation to Musical Beat Perception.

While rhythmic expectancies are thought to be at the base of beat perception in music, the extent to which stress patterns in speech are similarly represented and predicted during on-line language comprehension is debated. The temporal prediction of stress may be advantageous to speech processing, as stress patterns aid segmentation and mark new information in utterances. However, while linguistic stress patterns may be organized into hierarchical metrical structures similarly to musical meter, they do not typically present the same degree of periodicity. We review the theoretical background for the idea that stress patterns are predicted and address the following questions: First, what is the evidence that listeners can predict the temporal location of stress based on preceding rhythm? If they can, is it thanks to neural entrainment mechanisms similar to those utilized for musical beat perception? And lastly, what linguistic factors other than rhythm may account for the prediction of stress in natural speech? We conclude that while expectancies based on the periodic presentation of stresses are at play in some of the current literature, other processes are likely to affect the prediction of stress in more naturalistic, less isochronous speech. Specifically, aspects of prosody other than amplitude changes (e.g., intonation) as well as lexical, syntactic and information structural constraints on the realization of stress may all contribute to the probabilistic expectation of stress in speech.

Does attention control affect rhythmic temporal expectation?

Does attention control affect rhythmic temporal expectation?

Editorial: Overlap of Neural Systems for Processing Language and Music.

The relationship between musical training and speech perception has intrigued researchers in language and music for decades, from Bever and Chiarello's (1974) work emphasizing hemispheric specialization to Tallal and Gaab's (2006) findings of shared neural circuitry. Recent studies demonstrating neural overlap for processing speech and music, and enhanced speech perception and production in musicians, suggest that these regions may be inextricably intertwined (Sammler et al., 2007; Wong P.C. et al., 2007; Wong P. et al., 2007; Rogalsky et al., 2011; Schulze et al., 2011). Patel's OPERA hypothesis and Hickok and Poeppel's (2000, 2007) neuroanatomical models continue to evolve and guide this field of research. However, the extent of neural overlap between music and speech remains hotly debated (Norman-Haignere et al., 2015; Peretz et al., 2015), with surprisingly little empirical research exploring specific neural homologs and analogs. Emerging evidence suggests that shared processes likely exist throughout development, depend upon an individual's acoustic experiences, and are affected by developmental trajectories. Moreover, developing theories that address the neural and developmental interaction between music and language processing in conjunction with the broad availability of sophisticated tools for quantifying brain activity and dynamics offer the perfect opportunity for researchers to address these key empirical questions. Taken together, this field of research has begun to elucidate the complex dynamics of overlapping neural areas for processing language and music. This special issue highlights the development of this overlap in early childhood and explores how the interaction between language and musical training enhances cognitive functioning in adults. This E-Book comprises 10 opinion, perspective, and research papers that focus on the overlap of neural systems for processing language and music. Eight of these papers report original research and new findings that support overlapping neural systems for processing language and music. LaCroix et al. performed a meta-analysis of 171 neuroimaging studies to examine the role of context in processing music and language. Their findings suggest that observed neural overlaps for speech and music might be task-dependent. Fogel et al. developed a novel method for studying and quantifying predictions in musical tasks that is consistent with language tasks. Their melodic cloze probability task can be used to test computational models of melodic expectation and allows for a more precise examination of the relationship between predictive mechanisms in music and language. Using a garden-path design, Jung et al. demonstrated that rhythmic expectancy is crucial to the interaction of processing musical and linguistic syntax. Additionally, their findings support the incorporation of dynamic models of attentional entrainment into existing theories of musical and linguistic syntactical processing. Margulis et al. used the speech-to-song illusion to examine the role of pronunciation difficulty and temporal regularity. Their finding—that difficult to pronounce languages, not differing temporal intervals, elicited a stronger speech-to-song illusion—suggests a stronger speech representation for native and easy to pronounce languages. Miles et al. demonstrated that females have an advantage for recognizing familiar musical melodies. They believe this advantage is related to superior declarative memory, which may underlie the storage and knowledge of both the mental lexicon in language (e.g., Ullman, 2001) and some aspects of familiar melodies in music (Miranda and Ullman, 2007). Two papers report finding that musical training during development enhances literacy skills, including phonological awareness and reading fluency, via neural mechanisms for both language and music (Dege et al.; Gordon et al.). Moreover, Dege and colleagues provide evidence that music production and music perception are associated with multiple precursors of reading. Finally, Lolli et al. examined the effect of sound frequency on judgments of emotion in speech by congenital amusics. Using both high and low-pass filtered speech in a pitch discrimination and emotion identification task, their findings demonstrate the important role of low frequency information in identifying the emotional content of speech. In addition to these eight research papers there are two perspective and opinion papers that emphasize the affective and emotive commonalities between music and language (Lehmann and Paquette; Omigie). Lehmann and Paquette provide a neurobehavioral approach for examining cross-domain processing of musical and vocal emotions, suggesting that studying cochlear implant users may allow for a richer understanding of neural overlap between music and language. Omigie (2015) provides evolutionary evidence for shared underlying neural mechanisms for our emotive responses to music and literature. This E-Book provides a comprehensive snapshot of the research examining the complex overlap of neural systems for processing language and music. Both musical experience and training enhance the development of linguistic representations, emotion perception, and other cognitive skills. Furthermore, the research presented here contributes to current knowledge of neuroplastic reorganization and repair in clinical populations, and may aid in the design of new and more effective rehabilitative protocols.

Speech rhythm sensitivity and musical aptitude: ERPs and individual differences

This study investigated the electrophysiological markers of rhythmic expectancy during speech perception. In addition, given the large literature showing overlaps between cognitive and neural resources recruited for language and music, we considered a relation between musical aptitude and individual differences in speech rhythm sensitivity. Twenty adults were administered a standardized assessment of musical aptitude, and EEG was recorded as participants listened to sequences of four bisyllabic words for which the stress pattern of the final word either matched or mismatched the stress pattern of the preceding words. Words with unexpected stress patterns elicited an increased fronto-central mid-latency negativity. In addition, rhythm aptitude significantly correlated with the size of the negative effect elicited by unexpected iambic words, the least common type of stress pattern in English. The present results suggest shared neurocognitive resources for speech rhythm and musical rhythm.

Rhythmic Effects of Syntax Processing in Music and Language.

Music and language are human cognitive and neural functions that share many structural similarities. Past theories posit a sharing of neural resources between syntax processing in music and language (Patel, 2003), and a dynamic attention network that governs general temporal processing (Large and Jones, 1999). Both make predictions about music and language processing over time. Experiment 1 of this study investigates the relationship between rhythmic expectancy and musical and linguistic syntax in a reading time paradigm. Stimuli (adapted from Slevc et al., 2009) were sentences broken down into segments; each sentence segment was paired with a musical chord and presented at a fixed inter-onset interval. Linguistic syntax violations appeared in a garden-path design. During the critical region of the garden-path sentence, i.e., the particular segment in which the syntactic unexpectedness was processed, expectancy violations for language, music, and rhythm were each independently manipulated: musical expectation was manipulated by presenting out-of-key chords and rhythmic expectancy was manipulated by perturbing the fixed inter-onset interval such that the sentence segments and musical chords appeared either early or late. Reading times were recorded for each sentence segment and compared for linguistic, musical, and rhythmic expectancy. Results showed main effects of rhythmic expectancy and linguistic syntax expectancy on reading time. There was also an effect of rhythm on the interaction between musical and linguistic syntax: effects of violations in musical and linguistic syntax showed significant interaction only during rhythmically expected trials. To test the effects of our experimental design on rhythmic and linguistic expectancies, independently of musical syntax, Experiment 2 used the same experimental paradigm, but the musical factor was eliminated—linguistic stimuli were simply presented silently, and rhythmic expectancy was manipulated at the critical region. Experiment 2 replicated effects of rhythm and language, without an interaction. Together, results suggest that the interaction of music and language syntax processing depends on rhythmic expectancy, and support a merging of theories of music and language syntax processing with dynamic models of attentional entrainment.

Auditory rhythms entrain visual processes in the human brain: Evidence from evoked oscillations and event-related potentials

Temporal regularities in the environment are thought to guide the allocation of attention in time. Here, we explored whether entrainment of neuronal oscillations underpins this phenomenon. Participants viewed a regular stream of images in silence, or in-synchrony or out-of-synchrony with an unmarked beat position of a slow (1.3Hz) auditory rhythm. Focusing on occipital recordings, we analyzed evoked oscillations shortly before and event-related potentials (ERPs) shortly after image onset. The phase of beta-band oscillations in the in-synchrony condition differed from that in the out-of-synchrony and silence conditions. Additionally, ERPs revealed rhythm effects for a stimulus onset potential (SOP) and the N1. Both were more negative for the in-synchrony as compared to the out-of-synchrony and silence conditions and their amplitudes positively correlated with the beta phase effects. Taken together, these findings indicate that rhythmic expectations are supported by a reorganization of neural oscillations that seems to benefit stimulus processing at expected time points. Importantly, this reorganization emerges from global rhythmic cues, across modalities, and for frequencies significantly higher than the external rhythm. As such, our findings support the idea that entrainment of neuronal oscillations represents a general mechanism through which the brain uses predictive elements in the environment to optimize attention and stimulus perception.

The relevance of rhythmical alternation in language processing: An ERP study on English compounds

This study investigates the influence of rhythmic expectancies on language processing. It is assumed that language rhythm involves an alternation of strong and weak beats within a linguistic domain. Hence, in some contexts rhythmically induced stress shifts occur in order to comply with the Rhythm Rule. In English, this rule operates to prevent clashes of stressed adjacent syllables or lapses of adjacent unstressed syllables. While previous studies investigated effects on speech production and perception, this study focuses on brain responses to structures either obeying or deviating from this rule. Event-related potentials show that rhythmic regularity is relevant for language processing: rhythmic deviations evoked different ERP components reflecting the deviance from rhythmic expectancies. An N400 effect found for shifted items reflects higher costs in lexical processing due to stress deviation. The overall results disentangle lexical and rhythmical influences on language processing and complement the findings of previous studies on rhythmical processing.

Mechanisms of Implicit Timing: Cognitive and Electrophysiological Manifestations of Temporal Expectations Following Rhythmic Visual Input

Temporal processing is frequently performed implicitly, when temporal calculations are required in order to perform a task, whose output is not temporal in essence. One prevalent use of implicit timing is in the formation of temporal expectations, in which the timing of an upcoming event is predicted in order to prepare for its occurrence. The sources of this prediction can be “endogenous”, relying on memorizing the temporal interval between the expected event and a preceding reference event, measuring the time from the onset of the reference, and increasing preparedness when the expected interval elapses. However, temporal expectations can also be “exogenous”, namely based on extracting temporal information from the ongoing temporal dynamics of the input, rather than memory-dependent. One unique case of such temporal dynamics is when the input is isochronous (i.e., stimuli appear rhythmically, with fixed inter-onset-interval), such as in biological motion, speech, or music. Based on findings from auditory rhythms, it has been suggested that in such case, expectations are realized by synchronization of internal oscillators to the input periodicity, a process that encompasses both the representation of the interval and the application of expectation at the correct time. In two studies, we investigate temporal expectations created in visual rhythmic context, showing that they have unique cognitive and electrophysiological characteristics, and dissociating them from other types of temporal expectations.The first study establishes that exposure to rhythmic input is sufficient to form temporal expectations unintentionally, and dissociates them from intentional expectations. Targets were presented either in-phase or out-of-phase with rhythmically flickering coloured stimuli, in three conditions: 1. the rhythm was non-predictive; 2. the rhythm was non-predictive and temporal expectations were concurrently manipulated in a memory-based fashion using predictive colour cues; 3. the rhythm was predictive and intentionally used to form expectations. We found that even when expectations were intentionally based on colour cues, and reliance on the rhythm could be counter-productive, there was significant facilitation for targets that appeared incidentally in-phase with the rhythm. This modulation combined additively with the validity effect of colour. Congruently, the contingent negative variation (CNV), an EEG component present during target anticipation, was driven by the interval of the rhythm even when the rhythm was non-predictive. When intentionally using the rhythm to form expectations, the behavioural modulation was stronger. Additionally there was also modulation of the latency of the P3, a target-evoked EEG component related to stimulus evaluation, as well as desynchronization of pre-target alpha-band oscillations. Thus, the unintentional rhythmic temporal expectations are dissociated from both intentional rhythmic expectations and memory-based expectations.The second study examines whether intentional formation of rhythmic expectations involves more than application of endogenous expectations following learning of the repeating fixed interval in the rhythm. For this purpose, we compared behaviour and EEG activity in three conditions: 1. stimuli appeared a-rhythmically followed by the target; 2. stimuli appeared rhythmically followed by the target; 3. pairs of stimuli (S1, S2) were presented with a fixed interval within pair and a random interval between pairs with the second stimulus in the last pair being the target. In the latter two conditions, in most of the trials the target appeared at the cued interval (valid cue) and in few trials it appeared at a different interval (invalid). Reaction times were not different between predictive conditions for valid trials, but the validity effect was larger for rhythmic than for repeated interval condition. Concordantly, the CNV peak amplitudes were not different between predictive conditions. However, when targets were expected at the early interval but were omitted, the CNV remained at peak amplitude in the repeated interval condition, but returned to baseline in the rhythmic condition. Assuming that the CNV reflects the level of preparation, this implies that there is more focus in time around the expected time point in the rhythmic condition. Unlike the CNV, modulations of the P3 latency and desynchronization of pre-target alpha-band oscillations did not differ between rhythmic and repeated interval conditions. Thus, rhythmic expectations are unique in being more temporally accurate and more difficult to disengage from than memory-based expectations, and this effect is manifested in anticipatory activity rather than in post-target responses.

Language familiarity, expectation, and novice musical rhythm production.

The music of expert musicians reflects the speech rhythm of their native language. Here, we examine this effect in amateur and novice musicians. English- and French-speaking participants were both instructed to produce simple "English" and "French" tunes using only two keys on a keyboard. All participants later rated the rhythmic variability of English and French speech samples. The rhythmic variability of the "English" and "French" tunes that were produced reflected the perceived rhythmic variability in English and French speech samples. Yet, the pattern was different for English and French participants and did not correspond to the actual measured speech rhythm variability of the speech samples. Surprise recognition tests two weeks later confirmed that the music-speech relationship remained over time. The results show that the relationship between music and speech rhythm is more widespread than previously thought and that musical rhythm production by amateurs and novices is concordant with their rhythmic expectations in the perception of speech.

Cortical Activity during Perception of Musical Rhythm; Comparing Musicians and Non-musicians.

This study investigates the effects of musical training on brain activity to violations of rhythmic expectancies. We recorded behavioral and event-related brain potential (ERP) responses of musicians and non-musicians to discrepancies of rhythm between pairs of unfamiliar melodies based on Western classical rules. Rhythm deviations in the second melody involved prolongation of a note, thus creating a delay in the subsequent note; the duration of the second note was consequently shorter because the offset time was unchanged. In the first melody, on the other hand, the two notes were of equal duration. Musicians detected rhythm deviations significantly better than non-musicians. A negative auditory cortical potential in response to the omitted stimulus was observed at a latency of 150-250 ms from where the note should have been. There were no significant differences of amplitude or latency between musicians and non-musicians. In contrast, the N100 and P200 to the delayed note after the omission were significantly greater in amplitude in musicians compared to non-musicians especially in frontal and frontal-central areas. These findings indicate that long term musical training enhances brain cortical activities involved in processing temporal irregularities of unfamiliar melodies.

Differential Entrainment of Neuroelectric Delta Oscillations in Developmental Dyslexia

Oscillatory entrainment to the speech signal is important for language processing, but has not yet been studied in developmental disorders of language. Developmental dyslexia, a difficulty in acquiring efficient reading skills linked to difficulties with phonology (the sound structure of language), has been associated with behavioural entrainment deficits. It has been proposed that the phonological ‘deficit’ that characterises dyslexia across languages is related to impaired auditory entrainment to speech at lower frequencies via neuroelectric oscillations (<10 Hz, ‘temporal sampling theory’). Impaired entrainment to temporal modulations at lower frequencies would affect the recovery of the prosodic and syllabic structure of speech. Here we investigated event-related oscillatory EEG activity and contingent negative variation (CNV) to auditory rhythmic tone streams delivered at frequencies within the delta band (2 Hz, 1.5 Hz), relevant to sampling stressed syllables in speech. Given prior behavioural entrainment findings at these rates, we predicted functionally atypical entrainment of delta oscillations in dyslexia. Participants performed a rhythmic expectancy task, detecting occasional white noise targets interspersed with tones occurring regularly at rates of 2 Hz or 1.5 Hz. Both groups showed significant entrainment of delta oscillations to the rhythmic stimulus stream, however the strength of inter-trial delta phase coherence (ITC, ‘phase locking’) and the CNV were both significantly weaker in dyslexics, suggestive of weaker entrainment and less preparatory brain activity. Both ITC strength and CNV amplitude were significantly related to individual differences in language processing and reading. Additionally, the instantaneous phase of prestimulus delta oscillation predicted behavioural responding (response time) for control participants only.

Rhesus Monkeys (Macaca mulatta) Detect Rhythmic Groups in Music, but Not the Beat

It was recently shown that rhythmic entrainment, long considered a human-specific mechanism, can be demonstrated in a selected group of bird species, and, somewhat surprisingly, not in more closely related species such as nonhuman primates. This observation supports the vocal learning hypothesis that suggests rhythmic entrainment to be a by-product of the vocal learning mechanisms that are shared by several bird and mammal species, including humans, but that are only weakly developed, or missing entirely, in nonhuman primates. To test this hypothesis we measured auditory event-related potentials (ERPs) in two rhesus monkeys (Macaca mulatta), probing a well-documented component in humans, the mismatch negativity (MMN) to study rhythmic expectation. We demonstrate for the first time in rhesus monkeys that, in response to infrequent deviants in pitch that were presented in a continuous sound stream using an oddball paradigm, a comparable ERP component can be detected with negative deflections in early latencies (Experiment 1). Subsequently we tested whether rhesus monkeys can detect gaps (omissions at random positions in the sound stream; Experiment 2) and, using more complex stimuli, also the beat (omissions at the first position of a musical unit, i.e. the ‘downbeat’; Experiment 3). In contrast to what has been shown in human adults and newborns (using identical stimuli and experimental paradigm), the results suggest that rhesus monkeys are not able to detect the beat in music. These findings are in support of the hypothesis that beat induction (the cognitive mechanism that supports the perception of a regular pulse from a varying rhythm) is species-specific and absent in nonhuman primates. In addition, the findings support the auditory timing dissociation hypothesis, with rhesus monkeys being sensitive to rhythmic grouping (detecting the start of a rhythmic group), but not to the induced beat (detecting a regularity from a varying rhythm).

Identifying highly rhythmic stretches of speech with envelope-driven resonance analysis

This paper proposes envelope-driven resonance analysis as a technique for characterizing rhythmicity in speech, emphasizing the degree to which a brief stretch of speech creates a rhythmic expectancy. Most approaches to characterizing the rhythm of speech have utilized measurements derived from the durations of linguistically relevant units such as feet, syllables, or vocalic/consonantal intervals. Recently, alternative approaches have been developed which are based upon the amplitude envelope of the speech waveform after the waveform has been filtered to emphasize low-frequency oscillations associated with alternations between vowels and consonants. These approaches include spectral analysis and empirical mode decomposition of the envelope. The method explored here is resonance analysis, which utilizes a bank of resonators that differ in their characteristic resonant frequencies. The resonators are 2nd order dynamical systems analogous to driven, damped springs. The powers of the resonator amplitudes are analyzed during and subsequent to excitation by a speech amplitude envelope. The power and frequency distribution of the resonant response is used to identify highly rhythmic stretches of speech and characterize their spectral properties.

EEG Correlates of Song Prosody: A New Look at the Relationship between Linguistic and Musical Rhythm

Song composers incorporate linguistic prosody into their music when setting words to melody, a process called “textsetting.” Composers tend to align the expected stress of the lyrics with strong metrical positions in the music. The present study was designed to explore the idea that temporal alignment helps listeners to better understand song lyrics by directing listeners’ attention to instances where strong syllables occur on strong beats. Three types of textsettings were created by aligning metronome clicks with all, some or none of the strong syllables in sung sentences. Electroencephalographic recordings were taken while participants listened to the sung sentences (primes) and performed a lexical decision task on subsequent words and pseudowords (targets, presented visually). Comparison of misaligned and well-aligned sentences showed that temporal alignment between strong/weak syllables and strong/weak musical beats were associated with modulations of induced beta and evoked gamma power, which have been shown to fluctuate with rhythmic expectancies. Furthermore, targets that followed well-aligned primes elicited greater induced alpha and beta activity, and better lexical decision task performance, compared with targets that followed misaligned and varied sentences. Overall, these findings suggest that alignment of linguistic stress and musical meter in song enhances musical beat tracking and comprehension of lyrics by synchronizing neural activity with strong syllables. This approach may begin to explain the mechanisms underlying the relationship between linguistic and musical rhythm in songs, and how rhythmic attending facilitates learning and recall of song lyrics. Moreover, the observations reported here coincide with a growing number of studies reporting interactions between the linguistic and musical dimensions of song, which likely stem from shared neural resources for processing music and speech.

Disordered speech as a testing ground for listener learning and adaptation.

Speech production disorders arise from a wide variety of anatomical defects, neurological injury, and disease. Depending on the etiology, the speech manifestations may be quite regularized—in the same way that a foreign accent presents consistently—or highly variable with unpredictable speech interruption and degradation. Indeed, even within an individual speaker, one will find both relatively consistent and variable aspects of speech abnormality. A growing body of literature shows that listeners enjoy intelligibility gains with just brief auditory exposure to disordered speech that is accompanied by lexical feedback, and with repeated unsupervised exposures. These benefits have been shown to be both talker-specific and applicable to groups of people who presumably share relevant acoustic-perceptual speech features. While many of these studies have been designed with an eye toward clinical and communicative implications, there is much to be learned about the mechanisms underlying these perceptual improvements with naturally impoverished speech. The purpose of this presentation is to describe the use of clinical populations as a testing ground for hypotheses of listener learning and adaptation. Speech production disorders particularly lend themselves to the study of episodic memory, rhythmic expectancy, and talker-specific versus talker-general effects. [Work supported by NIDCD, NIH.]

Neural correlates of rhythmic expectancy

Temporal expectancy is thought to play a fundamental role in the perception of rhythm. This review summarizes recent studies that investigated rhythmic expectancy by recording neuroelectric activity with high temporal resolution during the presentation of rhythmic patterns. Prior event-related brain potential (ERP) studies have uncovered auditory evoked responses that reflect detection of onsets, offsets, sustains, and abrupt changes in acoustic properties such as frequency, intensity, and spectrum, in addition to indexing higher-order processes such as auditory sensory memory and the violation of expectancy. In our studies of rhythmic expectancy, we measured emitted responses – a type of ERP that occurs when an expected event is omitted from a regular series of stimulus events – in simple rhythms with temporal structures typical of music. Our observations suggest that middle-latency gamma band (20-60 Hz) activity (GBA) plays an essential role in auditory rhythm processing. Evoked (phase-locked) GBA occurs in the presence of physically presented auditory events and reflects the degree of accent. Induced (non-phase-locked) GBA reflects temporally precise expectancies for strongly and weakly accented events in sound patterns. Thus far, these findings support theories of rhythm perception that posit temporal expectancies generated by active neural processes.