Nonspeech Stimuli Research Articles

Individual differences in auditory perception predict learning of non-adjacent tone sequences in 3-year-olds.

Auditory processing of speech and non-speech stimuli oftentimes involves the analysis and acquisition of non-adjacent sound patterns. Previous studies using speech material have demonstrated (i) children's early emerging ability to extract non-adjacent dependencies (NADs) and (ii) a relation between basic auditory perception and this ability. Yet, it is currently unclear whether children show similar sensitivities and similar perceptual influences for NADs in the non-linguistic domain. We conducted an event-related potential study with 3-year-old children using a sine-tone-based oddball task, which simultaneously tested for NAD learning and auditory perception by means of varying sound intensity. Standard stimuli were A × B sine-tone sequences, in which specific A elements predicted specific B elements after variable × elements. NAD deviants violated the dependency between A and B and intensity deviants were reduced in amplitude. Both elicited similar frontally distributed positivities, suggesting successful deviant detection. Crucially, there was a predictive relationship between the amplitude of the sound intensity discrimination effect and the amplitude of the NAD learning effect. These results are taken as evidence that NAD learning in the non-linguistic domain is functional in 3-year-olds and that basic auditory processes are related to the learning of higher-order auditory regularities also outside the linguistic domain.

Precursors to infant sensorimotor synchronization to speech and non-speech rhythms: A longitudinal study.

Impaired sensorimotor synchronization (SMS) to acoustic rhythm may be a marker of atypical language development. Here, Motion Capture was used to assess gross motor rhythmic movement at six time points between 5- and 11 months of age. Infants were recorded drumming to acoustic stimuli of varying linguistic and temporal complexity: drumbeats, repeated syllables and nursery rhymes. Here we show, for the first time, developmental change in infants' movement timing in response to auditory stimuli over the first year of life. Longitudinal analyses revealed that whilst infants could not yet reliably synchronize their movement to auditory rhythms, infant spontaneous motor tempo became faster with age, and by 11 months, a subset of infants decelerate from their spontaneous motor tempo, which better accords with the incoming tempo. Further, infants became more regular drummers with age, with marked decreases in the variability of spontaneous motor tempo and variability in response to drumbeats. This latter effect was subdued in response to linguistic stimuli. The current work lays the foundation for using individual differences in precursors of SMS in infancy to predict later language outcomes. RESEARCH HIGHLIGHT: We present the first longitudinal investigation of infant rhythmic movement over the first year of life Whilst infants generally move more quickly and with higher regularity over their first year, by 11 months infants begin to counter this pattern when hearing slower infant-directed song Infant movement is more variable to speech than non-speech stimuli In the context of the larger Cambridge UK BabyRhythm Project, we lay the foundation for rhythmic movement in infancy to predict later language outcomes.

Spectral Grouping of Electrically Encoded Sound Predicts Speech-in-Noise Performance in Cochlear Implantees

ObjectivesCochlear implant (CI) users exhibit large variability in understanding speech in noise. Past work in CI users found that spectral and temporal resolution correlates with speech-in-noise ability, but a large portion of variance remains unexplained. Recent work on normal-hearing listeners showed that the ability to group temporally and spectrally coherent tones in a complex auditory scene predicts speech-in-noise ability independently of the audiogram, highlighting a central mechanism for auditory scene analysis that contributes to speech-in-noise. The current study examined whether the auditory grouping ability also contributes to speech-in-noise understanding in CI users.DesignForty-seven post-lingually deafened CI users were tested with psychophysical measures of spectral and temporal resolution, a stochastic figure-ground task that depends on the detection of a figure by grouping multiple fixed frequency elements against a random background, and a sentence-in-noise measure. Multiple linear regression was used to predict sentence-in-noise performance from the other tasks.ResultsNo co-linearity was found between any predictor variables. All three predictors (spectral and temporal resolution plus the figure-ground task) exhibited significant contribution in the multiple linear regression model, indicating that the auditory grouping ability in a complex auditory scene explains a further proportion of variance in CI users’ speech-in-noise performance that was not explained by spectral and temporal resolution.ConclusionMeasures of cross-frequency grouping reflect an auditory cognitive mechanism that determines speech-in-noise understanding independently of cochlear function. Such measures are easily implemented clinically as predictors of CI success and suggest potential strategies for rehabilitation based on training with non-speech stimuli.

Of words and whistles: Statistical learning operates similarly for identical sounds perceived as speech and non-speech

Statistical learning is an ability that allows individuals to effortlessly extract patterns from the environment, such as sound patterns in speech. Some prior evidence suggests that statistical learning operates more robustly for speech compared to non-speech stimuli, supporting the idea that humans are predisposed to learn language. However, any apparent statistical learning advantage for speech could be driven by signal acoustics, rather than the subjective perception per se of sounds as speech. To resolve this issue, the current study assessed whether there is a statistical learning advantage for ambiguous sounds that are subjectively perceived as speech-like compared to the same sounds perceived as non-speech, thereby controlling for acoustic features. We first induced participants to perceive sine-wave speech (SWS)—a degraded form of speech not immediately perceptible as speech—as either speech or non-speech. After this induction phase, participants were exposed to a continuous stream of repeating trisyllabic nonsense words, composed of SWS syllables, and then completed an explicit familiarity rating task and an implicit target detection task to assess learning. Critically, participants showed robust and equivalent performance on both measures, regardless of their subjective speech perception. In contrast, participants who perceived the SWS syllables as more speech-like showed better detection of individual syllables embedded in speech streams. These results suggest that speech perception facilitates processing of individual sounds, but not the ability to extract patterns across sounds. Our findings suggest that statistical learning is not influenced by the perceived linguistic relevance of sounds, and that it may be conceptualized largely as an automatic, stimulus-driven mechanism.

Studying rhythm processing in speech through the lens of auditory-motor synchronization

Continuous speech is organized into a hierarchy of rhythms. Accurate processing of this rhythmic hierarchy through the interactions of auditory and motor systems is fundamental to speech perception and production. In this mini-review, we aim to evaluate the implementation of behavioral auditory-motor synchronization paradigms when studying rhythm processing in speech. First, we present an overview of the classic finger-tapping paradigm and its application in revealing differences in auditory-motor synchronization between the typical and clinical populations. Next, we highlight key findings on rhythm hierarchy processing in speech and non-speech stimuli from finger-tapping studies. Following this, we discuss the potential caveats of the finger-tapping paradigm and propose the speech-speech synchronization (SSS) task as a promising tool for future studies. Overall, we seek to raise interest in developing new methods to shed light on the neural mechanisms of speech processing.

Language dependent cue-weighting of tone cues in speech and non-speech stimuli

Previous research on tone perception has identified several important pitch related cues including average pitch height (AH), contour, onset and offset, and the weighting of these cues has shown to be language dependent. However, since multiple pitch cues are covarying with each other, few studies have directly compared relative importance of these cues. It is also not clear whether there is a same ranking of cues in speech and non-speech stimuli. The current study aims to tease apart the relative role of each cue using AX discrimination. Four pairs of tone contrasts with minimal pitch differences were created. Tone contrasts within contour condition are two level tones with 7 Hz differences. Tone contrasts within AH, onset and offset conditions have one rising tone and one falling tone sharing the same AH, onset and offset respectively. If one cue is important, then when this cue is kept constant, variation in other cues should be hard to perceive. 48 Mandarin speakers and 48 Cantonese speakers were recruited. Results showed highest importance of AH for both Mandarin and Cantonese listeners and higher importance of contour (offset) than onset for Mandarin (Cantonese) listeners in speech stimuli. This ranking was not held for nonspeech stimuli.

Effects of audible bandwidth and stimulus modality on emotional responses to sound

A wide range of emotional responses is typical and important for overall well-being. Adults with hearing loss demonstrate a reduced range of emotional responses to non-speech sounds; their ratings of valence to pleasant and unpleasant sounds are less extreme than those of their peers with normal hearing. The purpose of this project was to explore the role of two potential explanations, limited audible bandwidth and emotional processing differences. Adults with normal hearing and sensorineural hearing loss rated their emotional responses (valence and arousal) to non-speech, affective sounds and pictures (in combination and in isolation). To explore the potential contribution of audible bandwidth, adults with normal hearing rated stimuli that were full bandwidth and also bandpass limited. In addition, adults with hearing loss were specifically recruited to include heterogeneous hearing loss configurations. Differences in ratings between auditory and visual stimuli provide insight into whether reduced emotional responses for adults with hearing loss are specific to audition or generalize to visual stimuli. Results demonstrate the importance of audition and audible bandwidth as key factors in emotional responses to pleasant, non-speech stimuli. These findings have important implications for designing interventions to support the listening needs of adults with hearing loss.

Japanese pitch-accent identification accuracy by children with autism spectrum disorder

A previous study demonstrated when Japanese speakers hear sine-wave speech carrying no fundamental frequency information, they rely on the first formant frequency for their pitch-accent perception, resulting in poor identification accuracy. However, when sine-wave speech is noise-vocoded, each formant becomes less audible, and Japanese speakers use other acoustic cues (e.g., duration and intensity), leading to better identification [Y. Shinohara, “Japanese pitch-accent perception of noise-vocoded sine-wave speech,” The 183rd Meeting of ASA, Nashville, TN (2022)]. The present study examined pitch-accent perception by children with autism spectrum disorder (ASD) for the same three stimulus types: natural recordings, sine-wave speech, and noise-vocoded sine-wave speech. Children with ASD often have enhanced auditory pitch perception for non-speech stimuli. However, since the present study used a pitch-accent word identification task, it was hypothesized that Japanese-speaking children with ASD would show less identification accuracy for all three stimulus types compared to age-matched typically developing children. Our preliminary results showed a significant difference in identification accuracy between the two groups. A detailed analysis of the results and the acoustic features of each stimulus type will be discussed.

Assessing Functional Brain Network Dynamics in Dyslexia from fNIRS Data.

Developmental dyslexia is characterized by a deficit of phonological awareness whose origin is related to atypical neural processing of speech streams. This can lead to differences in the neural networks that encode audio information for dyslexics. In this work, we investigate whether such differences exist using functional near-infrared spectroscopy (fNIRS) and complex network analysis. We have explored functional brain networks derived from low-level auditory processing of nonspeech stimuli related to speech units such as stress, syllables or phonemes of skilled and dyslexic seven-year-old readers. A complex network analysis was performed to examine the properties of functional brain networks and their temporal evolution. We characterized aspects of brain connectivity such as functional segregation, functional integration or small-worldness. These properties are used as features to extract differential patterns in controls and dyslexic subjects. The results corroborate the presence of discrepancies in the topological organizations of functional brain networks and their dynamics that differentiate between control and dyslexic subjects, reaching an Area Under ROC Curve (AUC) up to 0.89 in classification experiments.

Performance in an Audiovisual Selective Attention Task Using Speech-Like Stimuli Depends on the Talker Identities, But Not Temporal Coherence.

Audiovisual integration of speech can benefit the listener by not only improving comprehension of what a talker is saying but also helping a listener select a particular talker's voice from a mixture of sounds. Binding, an early integration of auditory and visual streams that helps an observer allocate attention to a combined audiovisual object, is likely involved in processing audiovisual speech. Although temporal coherence of stimulus features across sensory modalities has been implicated as an important cue for non-speech stimuli (Maddox et al., 2015), the specific cues that drive binding in speech are not fully understood due to the challenges of studying binding in natural stimuli. Here we used speech-like artificial stimuli that allowed us to isolate three potential contributors to binding: temporal coherence (are the face and the voice changing synchronously?), articulatory correspondence (do visual faces represent the correct phones?), and talker congruence (do the face and voice come from the same person?). In a trio of experiments, we examined the relative contributions of each of these cues. Normal hearing listeners performed a dual task in which they were instructed to respond to events in a target auditory stream while ignoring events in a distractor auditory stream (auditory discrimination) and detecting flashes in a visual stream (visual detection). We found that viewing the face of a talker who matched the attended voice (i.e., talker congruence) offered a performance benefit. We found no effect of temporal coherence on performance in this task, prompting an important recontextualization of previous findings.

Development, and validation of non-speech dichotic listening test

BackgroundClassic dichotic listening tests using speech stimuli result in right ear advantage, due to the dominant crossed pathway for speech and language. It is presumed that similar crossed dominance could exist for non-speech stimuli too. Hence, this is an attempt to develop and validate the dichotic non-speech test using environmental stimuli and explore the effect of focused attention on this test. Materials and methodThree lists of dichotic stimuli were created using these sounds with fifteen tokens in each list. Four professionals and non-professionals validated these materials. Normative estimation was obtained by administering the newly developed test on 70 adults and 70 children using a free-recall and forced-recall condition. ResultThe results showed a significant difference between the left ear and right scores where the left ear score was better than the right, depicting left ear advantage (LEA) for free recall condition in both groups. In the forced recall condition, LEA was not seen; rather the mean score was significantly higher in the attended ear, irrespective of the stimuli presented to the right or left ear. The test-retest reliability in free recall was good in both the ears and moderate for forced right ear conditions. ConclusionThe novel test consistently showed LEA with good reliability and can be used to assess the hemispheric asymmetry in normal subjects and also in test batteries for the clinical population.

Sex differences in neural processing of speech in neonates.

The large majority of studies shows that girls develop their language skills faster than boys in the first few years of life. Are girls born with this advantage in language development? The present study used fNIRS in neonates to investigate sex differences in neural processing of speech within the first days of life. We found that speech stimuli elicited significantly more brain activity than non-speech stimuli in both groups of male and female neonates. However, whereas girls showed significant HbO changes to speech stimuli only within the left hemisphere, boys exhibited simultaneous neural activations in both hemispheres, with a larger and more significant fronto-temporal cluster in the right hemisphere. Furthermore, in boys, the variation in time-to-peak latencies was considerably greater than in girls. These findings suggest an earlier maturation of language-related brain areas in girls and highlight the importance of sex-specific investigations of neural language networks in infants.

Auditory Pitch Perception in Autism Spectrum Disorder: A Systematic Review and Meta-Analysis.

Pitch plays an important role in auditory perception of music and language. This study provides a systematic review with meta-analysis to investigate whether individuals with autism spectrum disorder (ASD) have enhanced pitch processing ability and to identify the potential factors associated with processing differences between ASD and neurotypicals. We conducted a systematic search through six major electronic databases focusing on the studies that used nonspeech stimuli to provide a qualitative and quantitative assessment across existing studies on pitch perception in autism. We identified potential participant- and methodology-related moderators and conducted metaregression analyses using mixed-effects models. On the basis of 22 studies with a total of 464 participants with ASD, we obtained a small-to-medium positive effect size (g = 0.26) in support of enhanced pitch perception in ASD. Moreover, the mean age and nonverbal IQ of participants were found to significantly moderate the between-studies heterogeneity. Our study provides the first meta-analysis on auditory pitch perception in ASD and demonstrates the existence of different developmental trajectories between autistic individuals and neurotypicals. In addition to age, nonverbal ability is found to be a significant contributor to the lower level/local processing bias in ASD. We highlight the need for further investigation of pitch perception in ASD under challenging listening conditions. Future neurophysiological and brain imaging studies with a longitudinal design are also needed to better understand the underlying neural mechanisms of atypical pitch processing in ASD and to help guide auditory-based interventions for improving language and social functioning. https://doi.org/10.23641/asha.21614271.

Speech listening entails neural encoding of invisible articulatory features

Speech processing entails a complex interplay between bottom-up and top-down computations. The former is reflected in the neural entrainment to the quasi-rhythmic properties of speech acoustics while the latter is supposed to guide the selection of the most relevant input subspace. Top-down signals are believed to originate mainly from motor regions, yet similar activities have been shown to tune attentional cycles also for simpler, non-speech stimuli. Here we examined whether, during speech listening, the brain reconstructs articulatory patterns associated to speech production. We measured electroencephalographic (EEG) data while participants listened to sentences during the production of which articulatory kinematics of lips, jaws and tongue were also recorded (via Electro-Magnetic Articulography, EMA). We captured the patterns of articulatory coordination through Principal Component Analysis (PCA) and used Partial Information Decomposition (PID) to identify whether the speech envelope and each of the kinematic components provided unique, synergistic and/or redundant information regarding the EEG signals. Interestingly, tongue movements contain both unique as well as synergistic information with the envelope that are encoded in the listener's brain activity. This demonstrates that during speech listening the brain retrieves highly specific and unique motor information that is never accessible through vision, thus leveraging audio-motor maps that arise most likely from the acquisition of speech production during development.

Infants’ short-term memory for consonant–vowel syllables

This research examined whether the auditory short-term memory (STM) capacity for speech sounds differs from that for nonlinguistic sounds in 11-month-old infants. Infants were presented with streams composed of repeating sequences of either 2 or 4 syllables, akin to prior work by Ross-Sheehy and Newman (2015) using nonlinguistic musical instruments. These syllable sequences either stayed the same for every repetition (constant) or changed by one syllable each time it repeated (varying). Using the head-turn preference procedure, we measured infant listening time to each type of stream (constant vs varying and 2 vs 4 syllables). Longer listening to the varying stream was taken as evidence for STM because this required remembering all syllables in the sequence. We found that infants listened longer to the varying streams for 2-syllable sequences but not for 4-syllable sequences. This capacity limitation is comparable to that found previously for nonlinguistic instrument tones, suggesting that young infants have similar STM limitations for speech and nonspeech stimuli.

Nonspeech sounds are not all equally good at being nonspeech.

Perception of speech sounds has a long history of being compared to perception of nonspeech sounds, with rich and enduring debates regarding how closely they share similar underlying processes. In many instances, perception of nonspeech sounds is directly compared to that of speech sounds without a clear explanation of how related these sounds are to the speech they are selected to mirror (or not mirror). While the extreme acoustic variability of speech sounds is well documented, this variability is bounded by the common source of a human vocal tract. Nonspeech sounds do not share a common source, and as such, exhibit even greater acoustic variability than that observed for speech. This increased variability raises important questions about how well perception of a given nonspeech sound might resemble or model perception of speech sounds. Here, we offer a brief review of extremely diverse nonspeech stimuli that have been used in the efforts to better understand perception of speech sounds. The review is organized according to increasing spectrotemporal complexity: random noise, pure tones, multitone complexes, environmental sounds, music, speech excerpts that are not recognized as speech, and sinewave speech. Considerations are offered for stimulus selection in nonspeech perception experiments moving forward.

Cross-modal functional connectivity supports speech understanding in cochlear implant users.

Sensory deprivation can lead to cross-modal cortical changes, whereby sensory brain regions deprived of input may be recruited to perform atypical function. Enhanced cross-modal responses to visual stimuli observed in auditory cortex of postlingually deaf cochlear implant (CI) users are hypothesized to reflect increased activation of cortical language regions, but it is unclear if this cross-modal activity is "adaptive" or "mal-adaptive" for speech understanding. To determine if increased activation of language regions is correlated with better speech understanding in CI users, we assessed task-related activation and functional connectivity of auditory and visual cortices to auditory and visual speech and non-speech stimuli in CI users (n = 14) and normal-hearing listeners (n = 17) and used functional near-infrared spectroscopy to measure hemodynamic responses. We used visually presented speech and non-speech to investigate neural processes related to linguistic content and observed that CI users show beneficial cross-modal effects. Specifically, an increase in connectivity between the left auditory and visual cortices-presumed primary sites of cortical language processing-was positively correlated with CI users' abilities to understand speech in background noise. Cross-modal activity in auditory cortex of postlingually deaf CI users may reflect adaptive activity of a distributed, multimodal speech network, recruited to enhance speech understanding.

Audition controls the flow of visual time during multisensory perception

SummaryPrevious work addressing the influence of audition on visual perception has mainly been assessed using non-speech stimuli. Herein, we introduce the Audiovisual Time-Flow Illusion in spoken language, underscoring the role of audition in multisensory processing. When brief pauses were inserted into or brief portions were removed from an acoustic speech stream, individuals perceived the corresponding visual speech as “pausing” or “skipping”, respectively—even though the visual stimulus was intact. When the stimulus manipulation was reversed—brief pauses were inserted into, or brief portions were removed from the visual speech stream—individuals failed to perceive the illusion in the corresponding intact auditory stream. Our findings demonstrate that in the context of spoken language, people continually realign the pace of their visual perception based on that of the auditory input. In short, the auditory modality sets the pace of the visual modality during audiovisual speech processing.

Speech and non-speech measures of audiovisual integration are not correlated.

Many natural events generate both visual and auditory signals, and humans are remarkably adept at integrating information from those sources. However, individuals appear to differ markedly in their ability or propensity to combine what they hear with what they see. Individual differences in audiovisual integration have been established using a range of materials, including speech stimuli (seeing and hearing a talker) and simpler audiovisual stimuli (seeing flashes of light combined with tones). Although there are multiple tasks in the literature that are referred to as "measures of audiovisual integration," the tasks themselves differ widely with respect to both the type of stimuli used (speech versus non-speech) and the nature of the tasks themselves (e.g., some tasks use conflicting auditory and visual stimuli whereas others use congruent stimuli). It is not clear whether these varied tasks are actually measuring the same underlying construct: audiovisual integration. This study tested the relationships among four commonly-used measures of audiovisual integration, two of which use speech stimuli (susceptibility to the McGurk effect and a measure of audiovisual benefit), and two of which use non-speech stimuli (the sound-induced flash illusion and audiovisual integration capacity). We replicated previous work showing large individual differences in each measure but found no significant correlations among any of the measures. These results suggest that tasks that are commonly referred to as measures of audiovisual integration may be tapping into different parts of the same process or different constructs entirely.

Perceptual asymmetry between pitch peaks and valleys

Perceiving pitch in spoken utterances is an important part of the speech comprehension process. Previous studies on pitch perception concentrated on ‘peaks’, neglecting ‘valleys’, thus making it difficult to know how generalisable the findings are. In two experiments, we took a factorial approach and contrasted the pitch movement direction associated with a stressed syllable (‘peaks’ vs. ‘valleys’), the turning point shape (sharp turn, 25, 100 ms plateau), the accent height (varied in one-semitone steps), and pitch levels by shifting the whole contour up or down. We employed different sentences and also non-speech stimuli. Results showed that listeners were better at discriminating the pitch height between ‘peaks’ compared to ‘valleys’. In particular, ‘valleys’ in the low level posed challenges. A long pitch plateau forming a flat turn at a ‘peak’ or ‘valley’ made a ‘peak’ sound higher and a ‘valley’ lower compared to a sharp turn or short plateau of the same height. Similar results were observed across speech and non-speech stimuli. The perceptual asymmetry may lead listeners to allocate more attention to ‘peaks’ than ‘valleys’ in speech processing, while the effect of listeners’ native language deserves further investigation.