Asynchronous Cues Research Articles

Listeners can anticipate future segments before they identify the current one.

Speech unfolds rapidly over time, and the information necessary to recognize even a single phoneme may not be available simultaneously. Consequently, listeners must both integrate prior acoustic cues and anticipate future segments. Prior work on stop consonants and vowels suggests that listeners integrate asynchronous cues by partially activating lexical entries as soon as any information is available, and then updating this when later cues arrive. However, a recent study suggests that for the voiceless sibilant fricatives (/s/ and /ʃ/), listeners wait to initiate lexical access until all cues have arrived at the onset of the vowel. Sibilants also contain coarticulatory cues that could be used to anticipate the vowel upcoming. However, given these results, it is unclear if listeners could use them fast enough to speed vowel recognition. The current study examines anticipation by asking when listeners use coarticulatory information in the frication to predict the upcoming vowel. A visual world paradigm experiment found that listeners do not wait: they anticipate the vowel immediately from the onset of the frication, even as they wait several hundred milliseconds to identify the fricative. This finding suggests listeners do not strictly process phonemes in the order that they appear; rather the dynamics of language processing may be largely internal and only loosely coupled to the dynamics of the input.

What Are You Waiting For? Real-Time Integration of Cues for Fricatives Suggests Encapsulated Auditory Memory.

Speech unfolds over time, and the cues for even a single phoneme are rarely available simultaneously. Consequently, to recognize a single phoneme, listeners must integrate material over several hundred milliseconds. Prior work contrasts two accounts: (a) a memory buffer account in which listeners accumulate auditory information in memory and only access higher level representations (i.e., lexical representations) when sufficient information has arrived; and (b) an immediate integration scheme in which lexical representations can be partially activated on the basis of early cues and then updated when more information arises. These studies have uniformly shown evidence for immediate integration for a variety of phonetic distinctions. We attempted to extend this to fricatives, a class of speech sounds which requires not only temporal integration of asynchronous cues (the frication, followed by the formant transitions 150-350ms later), but also integration across different frequency bands and compensation for contextual factors like coarticulation. Eye movements in the visual world paradigm showed clear evidence for a memory buffer. Results were replicated in five experiments, ruling out methodological factors and tying the release of the buffer to the onset of the vowel. These findings support a general auditory account for speech by suggesting that the acoustic nature of particular speech sounds may have large effects on how they are processed. It also has major implications for theories of auditory and speech perception by raising the possibility of an encapsulated memory buffer in early auditory processing.

Word-learning performance of children with and without cochlear implants given synchronous and asynchronous cues

ABSTRACTThis study sought to evaluate the effects of synchronous and asynchronous auditory–visual cues on the word-learning performance of children with cochlear implants and children with normal hearing matched for chronological age. Children with cochlear implants (n = 9) who had worn the implant for less than one year and children matched for chronological age (n = 9) participated in rapid word-learning trials. Children with cochlear implants did not learn words in either the synchronous or asynchronous condition (U = 49.5, p = .99; d = 0.05). Children with normal hearing learned more words in the synchronous rather than asynchronous condition (U = 78.5, p = .04; d = 0.95). These findings represent a first step toward determining how task-level factors influence the lexical outcomes of children with cochlear implants.

Body Perception in Newborns

SummaryBody ownership and awareness has recently become an active topic of research in adults using paradigms such as the “rubber hand illusion” and “enfacement” [1–11]. These studies show that visual, tactile, postural, and anatomical information all contribute to the sense of body ownership in adults [12]. While some hypothesize body perception from birth [13], others have speculated on the importance of postnatal experience [14, 15]. Through studying body perception in newborns, we can directly investigate the factors involved prior to significant postnatal experience. To address this issue, we measured the looking behavior of newborns presented with visual-tactile synchronous and asynchronous cues, under conditions in which the visual information was either an upright (body-related stimulus; experiment 1) or inverted (non-body-related stimulus; experiment 2) infant face. We found that newborns preferred to look at the synchronous condition compared to the asynchronous condition, but only when the visual stimulus was body related. These results are in line with findings from adults and demonstrate that human newborns detect intersensory synchrony when related to their own bodies, consistent with the basic processes underlying body perception being present at birth.

Tracking the time course of phonetic cue integration during spoken word recognition

Speech perception requires listeners to integrate multiple cues that each contribute to judgments about a phonetic category. Classic studies of trading relations assessed the weights attached to each cue but did not explore the time course of cue integration. Here, we provide the first direct evidence that asynchronous cues to voicing (/b/ vs. /p/) and manner (/b/ vs. /w/) contrasts become available to the listener at different times during spoken word recognition. Using the visual world paradigm, we show that the probability of eye movements to pictures of target and of competitor objects diverge at different points in time after the onset of the target word. These points of divergence correspond to the availability of early (voice onset time or formant transition slope) and late (vowel length) cues to voicing and manner contrasts. These results support a model of cue integration in which phonetic cues are used for lexical access as soon as they are available.

Determinants of phrasing effects in rat serial pattern learning

Two experiments investigated how brief pauses introduced into serial patterns as phrasing cues would affect pattern learning in rats. In Experiment 1, a 24-element pattern consisted of eight 3-element chunks, whereas a 20-element pattern consisted of four 5-element chunks. In both patterns, 3.0-s temporal pauses placed at chunk boundaries (synchronous phrasing cues) facilitated learning compared to no phrasing. Cues "out of sync" with pattern structure (asynchronous phrasing cues) facilitated learning for the 24-element pattern and retarded learning for the 20-element pattern. Evidence suggested that in the latter case, 3.0-s pauses served as "blank" trials that induced rats to "skip" to the next serial position in sequence. In Experiment 2, shorter 0.5-s pauses served as phrasing cues in the 20-element pattern of Experiment 1. Synchronous short cues facilitated learning, whereas asynchronous phrasing cues had no effect. Furthermore, removal of synchronous cues produced deficits in performance on formerly cued trials, whereas removal of asynchronous cues had no effect. The results of Experiment 2 support the notion that in both experiments phrasing cues served as discriminative cues and indirectly suggest that rats are concurrently sensitive to pattern element cues, extra-sequence cues (such as phrasing cues), and to the relative timing of sequential events.

Effect of hearing loss on the ability to process synchronous and asynchronous cues for speech identification

This study investigated the effect of sensori-neural hearing loss on the ability to identify speech in noise for vowel-consonant-vowel tokens that were unprocessed, modulated synchronously across frequency, or modulated asynchronously across frequency. The modulator was a 20-Hz raised square wave. One goal of the study was to determine whether hearing-impaired listeners have a deficit in the ability to integrate asynchronous spectral information in the perception of speech. Speech tokens were presented at a high, fixed sound level and the level of a speech-shaped noise was changed adaptively to estimate the masked speech identification threshold. The performance of the hearing-impaired listeners was generally worse than that of the normal-hearing listeners, but the impaired listeners showed particularly poor performance in the synchronous modulation condition. The results did not suggest that the hearing-impaired listeners have a general deficit in the ability to integrate asynchronous spectral information in the perception of speech. Results will be discussed in terms of common mechanisms that might account for poor speech identification performance by hearing-impaired listeners when either the masking noise or the speech is synchronously modulated.

A lexical locus for the integration of asynchronous cues to voicing: An investigation with natural stimuli

Speech perception requires integrating multiple asynchronous acoustic cues. Two questions pertinent to the integration of VOT and vowel length in word recognition were examined. First, this study asked whether lexical activation is sufficient to integrate these cues or if a sublexical buffer is required. Second, the role of stimulus type was examined since studies have found different results for synthetic and natural stimuli (Shinn et al., 1985; Miller and Wayland, 1993; Utman, 1998). An eye tracking methodology that is sensitive to timing and small differences in activation was used (McMurray et al., 2002). Stimuli were recordings of words (e.g., ‘‘beach’’ and ‘‘peach’’) whose initial VOT was manipulated along a nine-step continuum using splicing. The length of the following vowel was modified to create long and short vowels. Participants selected the correct object from screens containing the target, competitor, and two filler objects while eye movements were monitored. A small vowel length effect was found on voicing perception. This effect occurred substantially later in the fixation record than the effect of VOT. Thus, effects of vowel length can be seen in natural speech. Further, these cues each affect lexical activation as soon as they are available; no sublexical integration is required.

Spectral integration of synchronous and asynchronous cues to consonant identification.

The goal of the present research was to determine how well observers utilize cues to consonant identification from different spectral regions that occur asynchronously as opposed to synchronously across frequency; such an ability would be useful for processing speech in the context of a spectro-temporally complex masker (e.g., competing speech). This was assessed by obtaining masked identification thresholds for VCV speech material, of the form /a/ C /a/, under various conditions of 10-Hz or 20-Hz square-wave amplitude modulation (AM). The speech tokens were filtered into 2, 4, 8, or 16 contiguous log-spaced frequency bands spanning 0.1 to 10 kHz. Bands were then modulated, with the pattern of that AM being either coherent across bands or 180 degrees out of phase for adjacent bands. In the out-of-phase conditions the odd-numbered bands had coherent phase AM and the even bands had coherent phase AM, but the AM pattern across these two subsets of bands were out of phase. Results from these two conditions, along with further conditions employing only modulated even- or odd-numbered bands, allowed performance to be compared between stimuli characterized by synchronous and asynchronous cues. Results indicate that observers are able to utilize asynchronously presented cues to consonant identification efficiently across a range of conditions.

Spectral integration of synchronous and asynchronous cues to consonant identification

When speech is presented at poor speech-to-masker ratios, the listener may momentarily receive some spectral regions of the signal at more favorable ratios than other regions. These optimal spectral regions may vary dynamically as a function of time. It is therefore of interest to determine how well listeners utilize cues from different spectral regions that occur asynchronously as opposed to synchronously across frequency. Here, we examined masked identification of consonants under various conditions of 10-Hz square-wave modulation of the speech material. Speech from 100–10<th>000 Hz was divided into either 2, 4, 8, or 16 contiguous bands. In one set of conditions, bands were amplitude modulated coherently, while in the other, even and odd bands were modulated 180 degrees out of phase. These conditions, along with further conditions employing only modulated even or odd bands, allowed performance to be compared between synchronous and asynchronous presentation. This approach is similar to a published checkerboard masking study in which the noise rather than the speech was modulated [Howard-Jones and Rosen, J. Acoust. Soc. Am. 93, 2915–2922]. Preliminary data suggest that many listeners are able to utilize asynchronous speech cues efficiently in all of the band number conditions examined. [Work supported by NIDCD (5R01DC000418).]

Visual processing beyond the inferior temporal visual cortex

Body ownership and awareness has recently become an active topic of research in adults using paradigms such as the “rubber hand illusion” and “enfacement” [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]. These studies show that visual, tactile, postural, and anatomical information all contribute to the sense of body ownership in adults [12]. While some hypothesize body perception from birth [13], others have speculated on the importance of postnatal experience [14, 15]. Through studying body perception in newborns, we can directly investigate the factors involved prior to significant postnatal experience. To address this issue, we measured the looking behavior of newborns presented with visual-tactile synchronous and asynchronous cues, under conditions in which the visual information was either an upright (body-related stimulus; experiment 1) or inverted (non-body-related stimulus; experiment 2) infant face. We found that newborns preferred to look at the synchronous condition compared to the asynchronous condition, but only when the visual stimulus was body related. These results are in line with findings from adults and demonstrate that human newborns detect intersensory synchrony when related to their own bodies, consistent with the basic processes underlying body perception being present at birth.