Sinewave Analogs Research Articles

Object labeling and disambiguation in 4-month-old infants.

The account that word learning starts in earnest during the second year of life, when infants have mastered the disambiguation skills, has recently been challenged by evidence that infants during the first year already know many common words. The preliminary ability to rapidly map and disambiguate linguistic labels was tested in Italian-speaking infants (N = 96, 47 boys; age = 4 and 6 months, eye tracking). Infants can rapidly map linguistic labels to objects and movements, and disambiguate the intended referents to novel words, but they fail with sinewave analogs. In hearing infants, mapping and disambiguation emerge early in development, and are flexible as to which visual referents infants are willing to map to linguistic labels, but may be constrained to linguistic sounds.

Mandatory dichotic integration of second-formant information: Contralateral sine bleats have predictable effects on consonant place judgments.

Speech-on-speech informational masking arises because the interferer disrupts target processing (e.g., capacity limitations) or corrupts it (e.g., intrusions into the target percept); the latter should produce predictable errors. Listeners identified the consonant in monaural buzz-excited three-formant analogues of approximant-vowel syllables, forming a place of articulation series (/w/-/l/-/j/). There were two 11-member series; the vowel was either high-front or low-back. Series members shared formant-amplitude contours, fundamental frequency, and F1+F3 frequency contours; they were distinguished solely by the F2 frequency contour before the steady portion. Targets were always presented in the left ear. For each series, F2 frequency and amplitude contours were also used to generate interferers with altered source properties-sine-wave analogues of F2 (sine bleats) matched to their buzz-excited counterparts. Accompanying each series member with a fixed mismatched sine bleat in the contralateral ear produced systematic and predictable effects on category judgments; these effects were usually largest for bleats involving the fastest rate or greatest extent of frequency change. Judgments of isolated sine bleats using the three place labels were often unsystematic or arbitrary. These results indicate that informational masking by interferers involved corruption of target processing as a result of mandatory dichotic integration of F2 information, despite the grouping cues disfavoring this integration.

A test of linguistic influences in the perceptual organization of speech.

Research in speech perception has explored how knowledge of a language influences phonetic perception. The current study investigated whether such linguistic influences extend to the perceptual (sequential) organization of speech. Listeners heard sinewave analogs of word pairs (e.g., loose seam, which contains a single [s] frication but is perceived as two /s/ phonemes) cycle continuously, which causes the stimulus to split apart into foreground and background percepts. They had to identify the foreground percept when the stimuli were heard as nonspeech and then again when heard as speech. Of interest was how grouping changed across listening condition when [s] was heard as speech or as a hiss. Although the section of the signal that was identified as the foreground differed little across listening condition, a strong bias to perceive [s] as forming the onset of the foreground was observed in the speech condition (Experiment 1). This effect was reduced in Experiment 2 by increasing the stimulus repetition rate. Findings suggest that the sequential organization of speech arises from the interaction of auditory and linguistic processes, with the former constraining the latter.

Acoustic source characteristics, across-formant integration, and speech intelligibility under competitive conditions.

An important aspect of speech perception is the ability to group or select formants using cues in the acoustic source characteristics—for example, fundamental frequency (F0) differences between formants promote their segregation. This study explored the role of more radical differences in source characteristics. Three-formant (F1+F2+F3) synthetic speech analogues were derived from natural sentences. In Experiment 1, F1+F3 were generated by passing a harmonic glottal source (F0 = 140 Hz) through second-order resonators (H1+H3); in Experiment 2, F1+F3 were tonal (sine-wave) analogues (T1+T3). F2 could take either form (H2 or T2). In some conditions, the target formants were presented alone, either monaurally or dichotically (left ear = F1+F3; right ear = F2). In others, they were accompanied by a competitor for F2 (F1+F2C+F3; F2), which listeners must reject to optimize recognition. Competitors (H2C or T2C) were created using the time-reversed frequency and amplitude contours of F2. Dichotic presentation of F2 and F2C ensured that the impact of the competitor arose primarily through informational masking. In the absence of F2C, the effect of a source mismatch between F1+F3 and F2 was relatively modest. When F2C was present, intelligibility was lowest when F2 was tonal and F2C was harmonic, irrespective of which type matched F1+F3. This finding suggests that source type and context, rather than similarity, govern the phonetic contribution of a formant. It is proposed that wideband harmonic analogues are more effective informational maskers than narrowband tonal analogues, and so become dominant in across-frequency integration of phonetic information when placed in competition.

Music expertise shapes audiovisual temporal integration windows for speech, sinewave speech, and music.

This psychophysics study used musicians as a model to investigate whether musical expertise shapes the temporal integration window for audiovisual speech, sinewave speech, or music. Musicians and non-musicians judged the audiovisual synchrony of speech, sinewave analogs of speech, and music stimuli at 13 audiovisual stimulus onset asynchronies (±360, ±300 ±240, ±180, ±120, ±60, and 0 ms). Further, we manipulated the duration of the stimuli by presenting sentences/melodies or syllables/tones. Critically, musicians relative to non-musicians exhibited significantly narrower temporal integration windows for both music and sinewave speech. Further, the temporal integration window for music decreased with the amount of music practice, but not with age of acquisition. In other words, the more musicians practiced piano in the past 3 years, the more sensitive they became to the temporal misalignment of visual and auditory signals. Collectively, our findings demonstrate that music practicing fine-tunes the audiovisual temporal integration window to various extents depending on the stimulus class. While the effect of piano practicing was most pronounced for music, it also generalized to other stimulus classes such as sinewave speech and to a marginally significant degree to natural speech.

Toddlers' comprehension of noise-vocoded speech and sine-wave analogs to speech

A great deal of research has investigated listeners’ ability to compensate for degraded speech signals such as noise-vocoded speech (a signal with reduced spectral structure but intact amplitude envelope information) and sine-wave analogs to speech (a signal that maintains the global dynamic spectral structure of the signal at the expense of amplitude envelope information). Nittrouer and colleagues found developmental changes in the ability to comprehend such signals, reporting that while adults perform more accurately with sine-wave analogs than with noise-vocoded speech, school-aged children show the opposite pattern [e.g., Nittrouer Lowenstein and Packer (2009)]. In a series of studies, we tested toddler’s comprehension of these degraded signals. Twenty-seven-month-old children saw two images on each trial (e.g., cat, dog), and heard a voice instructing them which image to look at (“Find the cat!”). Sentences were presented either in full speech or were degraded. Toddlers (n = 24 per condition) looked at the appropriate object equally long with vocoded speech of 24 channels (60.2%) or 8 channels (62.4%) as with full speech (62.6%), but performed barely above chance with 4 channels (53.6%) and at chance for 2 channels (49.8%). Preliminary results suggest that performance with sine-wave analogs is poorer than 8-channel vocoded speech (56.1%), but testing is ongoing.

Evidence for atypical categorical speech perception in Williams syndrome

Although language processing has been described as being a relatively ‘spared’ cognitive function in individuals presenting with Williams syndrome (7q11.23 microdeletion; WS), some studies suggest possible alterations at the level of input speech processing. We explored categorical speech perception and non-speech auditory perception in six participants with WS, as well as in chronological age-matched or reading age-matched control groups. Categorical speech perception was explored for the b-d speech sound continuum, where speech sounds vary in spectral acoustic features, and for the d-t speech sound continuum, where speech sounds vary in temporal acoustic features. Non-speech perception was explored for sine-wave analogues of the b-d continuum, which are not identified as speech stimuli. We observed a significantly increased ability in WS participants to detect subtle acoustic changes between adjacent stimuli of the b-d or the d-t continuum outside the phoneme boundary, when control participants showed absence of discrimination. This is the first study to provide evidence for an atypical sensitivity towards subtle acoustic variations during speech and non-speech auditory analysis in WS. Implications for phonological processing and reading acquisition are discussed.

The perceptual organization of sine-wave speech under competitive conditions

Speech comprises dynamic and heterogeneous acoustic elements, yet it is heard as a single perceptual stream even when accompanied by other sounds. The relative contributions of grouping "primitives" and of speech-specific grouping factors to the perceptual coherence of speech are unclear, and the acoustical correlates of the latter remain unspecified. The parametric manipulations possible with simplified speech signals, such as sine-wave analogues, make them attractive stimuli to explore these issues. Given that the factors governing perceptual organization are generally revealed only where competition operates, the second-formant competitor (F2C) paradigm was used, in which the listener must resist competition to optimize recognition [Remez, R. E., et al. (1994). Psychol. Rev. 101, 129-156]. Three-formant (F1+F2+F3) sine-wave analogues were derived from natural sentences and presented dichotically (one ear=F1+F2C+F3; opposite ear=F2). Different versions of F2C were derived from F2 using separate manipulations of its amplitude and frequency contours. F2Cs with time-varying frequency contours were highly effective competitors, regardless of their amplitude characteristics. In contrast, F2Cs with constant frequency contours were completely ineffective. Competitor efficacy was not due to energetic masking of F3 by F2C. These findings indicate that modulation of the frequency, but not the amplitude, contour is critical for across-formant grouping.

Why voice melody alone cannot explain neonates’ preference for speech

This is a response to the commentary on Vouloumanos and Werker (2007) by Rosen and Iverson (2007). Are humans born with a bias for listening to the vocal-izations of their species? In Vouloumanos and Werker(2007, this issue), we present data demonstrating thatfrom birth, the human infant prefers listening to speech,compared with non-speech sounds that mimic spectraland temporal properties of speech. Rosen and Iverson(2007, this issue) criticize this interpretation, first argu-ing that the preference we have shown is based on voicemelody rather than speech per se; second, they arguethat such a voice melody preference likely stems fromprenatal learning, rather than from an innate bias – aclaim we didn’t make in Vouloumanos and Werker, butthat is addressed by new data we present here.Turning to Rosen and Iverson’s first point, althoughwe agree that a voice melody account of newborns’preference for speech is not altogether impossible, wefind it implausible. Voice melody, or pitch, is the subjectivehighness or lowness of a sound as perceived by thehuman ear. Although pitch extraction is not fully under-stood (e.g. Patel & Balaban, 2001), in natural speech,pitch is generally perceived as the fundamental fre-quency (F0) of an utterance, which is the frequency withwhich a particular speaker’s vocal folds vibrate (typicallyaround 200 Hz for a woman’s voice). Because of reson-ance properties of sound, F0 is reflected in ‘harmonics’at integer multiples of F0 (e.g. an F0 of 150Hz (itself thefirst harmonic), will have harmonics at 300 Hz, 450 Hz,600 Hz, etc.), which contribute to the perception of pitchif, for example, F0 is missing. Research on infant pitchperception is limited, but has shown that 7-month-oldinfants demonstrate some adult-like characteristics intheir perception of pitch (Montgomery & Clarkson, 1997).Even at this age, however, there is considerable variationin individual infants’ abilities to recover pitch when F0is missing (Clarkson, 1992). Pitch extraction in youngerinfants is currently poorly understood but is believed todiffer from adult pitch perception (Bundy, Colombo S Clarkson, 1992). Though neonates aresensitive to pitch contours, discriminating, for example,high-low pitch from low-high pitch in bimoraic stimuli(Nazzi, Floccia & Bertoncini, 1998), the mechanism ofpitch extraction in neonates has not been investigated.To examine neonates’ preference for speech, the non-speech sounds we used were a variant on sine-waveanalogues (SWA) of speech (Remez, Rubin, Pisoni &Carrell, 1981). SWA consist of time-varying sinusoidalwaves, or sine waves, that track the centre frequencies ofthe energy bands (formants) of natural speech to reproducethe changes in these frequency peaks across time. SWAare typically composed of three sinusoidal waves thatreproduce the changes in the first three formants of speechso adroitly that under the right circumstances, adultlisteners perceive SWA as intelligible (if weird) speech(Remez et al., 1981). At stake here is which componentof our SWA conveyed the perception of pitch. Rosenand Iverson suggest that because the first formant (F1)is usually heard as conveying pitch in SWA, even withthe addition of F0 (Remez & Rubin, 1984), F1 is likelyto convey perceived pitch in our stimuli as well, andthus, the voice melody perceived in our SWA is less salientcompared to that in the speech set. We would arguethat the F0 component in our SWA was salient, and thatit, rather than F1, accounted for the perceived pitch. Thekey lies in the construction of the stimuli by SonyaBird and Guy Carden, of the University of Victoria, andthe University of British Columbia, respectively. Whilecreating the SWA, they found that the first three formants(F1, F2, and F3) were virtually identical across themultiple natural speech tokens. For this reason, theyselected one representative set of the first three formants

Constructing adequate non‐speech analogues: what is special about speech anyway?

Vouloumanos and Werker (2007) claim that human neonates have a (possibly innate) bias to listen to speech based on a preference for natural speech utterances over sine-wave analogues. We argue that this bias more likely arises from the strikingly different saliency of voice melody in the two kinds of sounds, a bias that has already been shown to be learned pre-natally. Possible avenues of research to address this crucial issue are proposed, based on a consideration of the distinctive acoustic properties of speech. This is a commentary on Vouloumanos and Werker (2007).

Top-down processes during auditory phoneme categorization in dyslexia: A PET study

While persistence of subtle phonological deficits in dyslexic adults is well documented, deficit of categorical perception of phonemes has received little attention so far. We studied learning of phoneme categorization during an activation H(2)O(15) PET experiment in 14 dyslexic adults and 16 normal readers with similar age, handedness and performance IQ. Dyslexic subjects exhibited typical, marked impairments in reading and phoneme awareness tasks. During the PET experiment, subjects performed a discrimination task involving sine wave analogues of speech first presented as pairs of electronic sounds and, after debriefing, as syllables /ba/ and /da/. Discrimination performance and brain activation were compared between the acoustic mode and the speech mode of the task which involved physically identical stimuli; signal changes in the speech mode relative to the acoustic mode revealed the neural counterparts of phonological top-down processes that are engaged after debriefing. Although dyslexic subjects showed good abilities to learn discriminating speech sounds, their performance remained lower than those of normal readers on the discrimination task over the whole experiment. Activation observed in the speech mode in normal readers showed a strongly left-lateralized pattern involving the superior temporal, inferior parietal and inferior lateral frontal cortex. Frontal and parietal subparts of these left-sided regions were significantly more activated in the control group than in the dyslexic group. Activations in the right frontal cortex were larger in the dyslexic group than in the control group for both speech and acoustic modes relative to rest. Dyslexic subjects showed an unexpected large deactivation in the medial occipital cortex for the acoustic mode that may reflect increased effortful attention to auditory stimuli.

The inhibition of stuttering via the presentation of natural speech and sinusoidal speech analogs

Sensory signals containing speech or gestural (articulatory) information (e.g., choral speech) have repeatedly been found to be highly effective inhibitors of stuttering. Sine wave analogs of speech consist of a trio of changing pure tones representative of formant frequencies. They are otherwise devoid of traditional speech cues, yet have proven to evoke consistent linguistic percepts in listeners. Thus, we investigated the potency of sinusoidal speech for inhibiting stuttering. Ten adults who stutter read while listening to (a) forward-flowing natural speech; (b) forward-flowing sinusoid analogs of natural speech; (c) reversed natural speech; (d) reversed sinusoid analogs of natural speech; and (e) a continuous 1000 Hz pure tone. The levels of stuttering inhibition achieved using the sinusoidal stimuli were potent and not significantly different from those achieved using natural speech (∼50% in forward conditions and ∼25% in the reversed conditions), suggesting that the patterns of undulating pure tones are sufficient to endow sinusoidal sentences with ‘quasi-gestural’ qualities. These data highlight the sensitivity of a specialized ‘phonetic module’ for extracting gestural information from sensory stimuli. Stuttering inhibition is thought to occur when perceived gestural information facilitates fluent productions via the engagement of mirror neurons (e.g., in Broca's area), which appear to play a crucial role in our ability to perceive and produce speech.

Spectro-temporal processing during speech perception involves left posterior auditory cortex

This functional magnetic resonance imaging study investigates the neural underpinnings of spectro-temporal integration during speech perception. Participants performed an auditory discrimination task on a set of sine-wave analogues that could be perceived as either nonspeech or speech. Behavioural results revealed a difference in the processing mode; spectro-temporal integration occurred during speech perception, but not when stimuli were perceived as nonspeech. In terms of neuroimaging, we observed an activation increase in the left posterior primary and secondary auditory cortex, namely Heschl's gyrus and planum temporale encroaching onto the superior temporal sulcus, reflecting a shift from auditory to speech perception. This finding demonstrates that the left posterior superior temporal lobe is essential for spectro-temporal processing during speech perception.

Perceptual overshoot in listeners with cochlear implants

Perceptual overshoot (PO), a phenomenon in which the boundary for perceived vowel categories shifts as a result of preceding formant transitions, has been demonstrated previously with synthetic vowels in CV contexts in normal-hearing individuals. In one of the previous studies by the author, the same phenomenon was tested with sinewave analogs that mimicked the synthetic vowels. The results demonstrated that PO could be elicited for sinewave analogs after training on categorization of sinewave steady states. These findings suggest that PO may be partly mediated by general processes in the auditory system. In the current study, subjects with cochlear implants were presented with synthetic vowel and sinewave continua with and without transitions. It was predicted that PO would be greatly diminished or absent for this population because of the degraded nature of the spectral input. Implications of the results for theories of speech perception will be presented.

Role of experience in eliciting perceptual overshoot with sine waves

Previous studies in vowel perception have demonstrated perceptual overshoot (PO) with vowels in CVC contexts. The first experiment investigated the elicitation of PO with speech and nonspeech signals. Boundary estimates were obtained for four continua: (1) steady-state synthetic vowels; (2) synthetic vowel transitions in /wV/ context; (3) sine wave analogs mimicking the steady-state vowels; and (4) sine wave analogs mimicking the transition continuum. Results replicated the findings of previous studies by demonstrating PO with synthetic vowels; however, PO was not elicited with the sine wave analogs. This led to a second experiment, where the role of training on the elicitation of PO with sine wave analogs was investigated. Subjects from the first experiment were divided into two groups. One group was trained to categorize the steady-state sine wave analogs based on their synthetic vowel boundary. Training occurred over 3–6 1-h sessions until subjects achieved 90%-correct identification. The control group received no training. Post-training boundary estimates were obtained for the four original continua from both groups. The results indicated that PO was now elicited with the sine wave analogs for the trained group, but not for the control group. Further implications of these findings will be discussed.

Priming effects in speech and nonspeech modes of perception

A phonemic perception was assumed to be a process in which representation of speech signals is matched to phonemic representation in long-term memory. Since stimulation is supposed to activate relevant memory representation, priming effects were compared between speech sounds and their sinewave analogs, either in the speech mode or nonspeech mode of perception. An analog target imitating syllables /ba/ or /ga/ in Session 1 or their original speech target in Session 2 was preceded by one of 5 primers, which were these 4 target sounds, and a 1,000 Hz pure tone, with one of 6 interstimulus intervals (ISIs). In Session 1, half of the participants were asked to identify analogs as ill-synthesized syllables, “BA” or “GA,” and the other half were asked to label them as computer sounds, named “A” or “B.” In Session 2, all of them identified original syllables. Regardless of target types and ISIs, subjects given speech instructions showed faster identification of targets preceded by phonemically identical primers, while subjects given nonspeech instructions identified targets faster only when targets were preceded by physically identical primers. These results support that speech sounds and their analogs are accessible to common memory representation.

Perceptual discrimination of speech sounds in developmental dyslexia.

Experiments previously reported in the literature suggest that people with dyslexia have a deficit in categorical perception. However, it is still unclear whether the deficit is specific to the perception of speech sounds or whether it more generally affects auditory function. In order to investigate the relationship between categorical perception and dyslexia, as well as the nature of this categorization deficit, speech specific or not, the discrimination responses of children who have dyslexia and those of average readers to sinewave analogues of speech sounds were compared. These analogues were presented in two different conditions, either as nonspeech whistles or as speech sounds. Results showed that children with dyslexia are less categorical than average readers in the speech condition, mainly because they are better at discriminating acoustic differences between stimuli belonging to the same category. In the nonspeech condition, discrimination was also better for children with dyslexia, but differences in categorical perception were less clear-cut. Further, the location of the categorical boundary on the stimulus continuum differed between speech and nonspeech conditions. As a whole, this study shows that categorical deficit in children with dyslexia results primarily from an increased perceptibility of within-category differences and that it has a speech-specific component. These findings may have profound implications for learning and re-education.

Must sine wave analogs be perceived as implicit speech to be perceived categorically?

Sawusch and Gagnon [J. Exp. Psychol., HPP 21, 635–652 (1995)] obtained categorical responses to /la/–/ra/ sine wave analogs when listeners who heard the stimuli as nonspeech were frequently reminded of F3’s range or extremes. This result contrasts with responses to sine wave /la/–/ra/ obtained by Best et al. [Percept. Psychophys. 45, 237–250 (1989)] and responses to sine wave /wa/–/ja/ obtained by Johnson and Ralston [Phonetica 51, 195–209 (1994)]. Only those listeners who heard the analogs as (distorted) speech responded categorically. Instead of comparing speech and nonspeech listeners, the experiments tried to create an implicit expectation that sine wave analogs are (distorted) speech by playing listeners speech first. Listeners identified and discriminated members of sine wave /la/–/ra/ after identifying and discriminating either members of the original /la/–/ra/ speech continuum or of the same sine wave continuum. Other groups of listeners heard speech or sine wave /wa/–/ja/ before the sine wave /la/–/ra/; their performance allows us to separate acoustic from type similarity. If after hearing speech, listeners hear the analogs as implicit speech, they should perceive them more categorically than listeners who hear analogs first. Results show instead that listeners responses are as strongly categorical to the sine wave /la/–/ra/ in the second phase after all preceding stimulus types, even though the analogs are not heard as speech. Categorical perception thus does not depend on an implicit speech percept, nor are frequent reminders required to elicit it to sine wave analogs. [Work supported by NIH.]

Speech Perception Deficits in Poor Readers: Auditory Processing or Phonological Coding?

Poor readers are inferior to normal-reading peers in aspects of speech perception. Two hypotheses have been proposed to account for their deficits: (i) a speech-specific failure in phonological representation and (ii) a general deficit in auditory “temporal processing,” such that they cannot easily perceive the rapid spectral changes of formant transitions at the onset of stop-vowel syllables. To test these hypotheses, two groups of second-grade children (20 “good readers,” 20 “poor readers”), matched for age and intelligence, were selected to differ significantly on a /ba/–/da/ temporal order judgment (TOJ) task, said to be diagnostic of a temporal processing deficit. Three experiments then showed that the groups did not differ in: (i) TOJ when /ba/ and /da/ were paired with more easily discriminated syllables (/ba/–/sa/, /da/–/∳a/); (ii) discriminating nonspeech sine wave analogs of the second and third formants of /ba/ and /da/; (iii) sensitivity to brief transitional cues varying along a synthetic speech continuum. Thus, poor readers’ difficulties with /ba/–/da/ reflected perceptual confusion between phonetically similar, though phonologically contrastive, syllables rather than difficulty in perceiving rapid spectral changes. The results are consistent with a speech-specific, not a general auditory, deficit.

Reading disability: A deficit in the processing of rapid spectral changes or in phonological coding?

Reading‐impaired children have been said to suffer from an auditory deficit, indexed by difficulties in processing formant transitions [P. Tallal, Brain & Lang. 9, 182–198 (1980)]. Having established that poor readers (n=20) made significantly more errors than good readers (n=20) on discrimination between stop‐vowel syllables, contrasting in initial F2 and F3 transitions (/ba/ and /da/), but not between nonspeech sinewave analogs of F2 and F3, the present study compared the same groups on identification of (i) a /sei‐stei/ synthetic continuum, where F1 transition cued the contrast, and (ii) two hybrid /su–sh/ continua, each icative series by the formant transitions of naturally spoken vowels. Poor readers did not differ from good readers in phoneme boundaries, but did exhibit shallower identification functions, significantly so on the fricative continua. The results do not support the hypothesized auditory deficit: Less consistent discrimination or identification by poor readers on speech tasks but not on the nonspeech control task, suggests phonological rather than auditory difficulties.