Nonlinguistic Sounds Research Articles

Silent lipreading and covert speech production suppress processing of non-linguistic sounds in auditory cortex

Previous studies have suggested that speech motor system mediates suppression by silent lipreading of electromagnetic auditory cortex responses to pure tones at about 100 ms from sound onset. We used sparse sampling functional magnetic resonance imaging (fMRI) at 3 Tesla to map auditory-cortex foci of suppressant effects during silent lipreading and covert self-production. Streams of video clips were presented simultaneously with 1/3 octave noise bursts centered at 250 Hz (low frequency, LF) or 2000 Hz (mid-frequency, MF), or during no auditory stimulation. In different conditions, the subjects were a) to press a button whenever they lipread the face articulate the same consecutive Finnish vowels /a/, /i/, /o/, and /y/, b) covertly selfproducing vowels while viewing still face image, or c) to press a button whenever a circle pictured on top of the lips expanded into oval shape of the same orientation twice in a row. The regions of interest (ROIs) within the superior temporal lobes of each hemisphere were defined by contrasting MF and LF stimulation against silence. Contrasting the nonlinguistic (i.e., expanding circle) vs. linguistic (i.e., lipreading and covert self-production) conditions within these ROIs showed significant suppression of hemodynamic activity to MF sounds in the linguistic condition in left hemisphere first transverse sulcus (FTS) and right hemisphere superior temporal gyrus (STG) lateral to Heschl's sulcus (HS). These findings suggest that the speech motor system mediates suppression of auditory-cortex processing of non-linguistic sounds during silent lipreading and covert self-production in left hemisphere FST and right hemisphere STG lateral to HS.

A taste for words and sounds: a case of lexical-gustatory and sound-gustatory synesthesia

Gustatory forms of synesthesia involve the automatic and consistent experience of tastes that are triggered by non-taste related inducers. We present a case of lexical-gustatory and sound-gustatory synesthesia within one individual, SC. Most words and a subset of non-linguistic sounds induce the experience of taste, smell and physical sensations for SC. SC's lexical-gustatory associations were significantly more consistent than those of a group of controls. We tested for effects of presentation modality (visual vs. auditory), taste-related congruency, and synesthetic inducer-concurrent direction using a priming task. SC's performance did not differ significantly from a trained control group. We used functional magnetic resonance imaging to investigate the neural correlates of SC's synesthetic experiences by comparing her brain activation to the literature on brain networks related to language, music, and sound processing, in addition to synesthesia. Words that induced a strong taste were contrasted to words that induced weak-to-no tastes (“tasty” vs. “tasteless” words). Brain activation was also measured during passive listening to music and environmental sounds. Brain activation patterns showed evidence that two regions are implicated in SC's synesthetic experience of taste and smell: the left anterior insula and left superior parietal lobe. Anterior insula activation may reflect the synesthetic taste experience. The superior parietal lobe is proposed to be involved in binding sensory information across sub-types of synesthetes. We conclude that SC's synesthesia is genuine and reflected in her brain activation. The type of inducer (visual-lexical, auditory-lexical, and non-lexical auditory stimuli) could be differentiated based on patterns of brain activity.

Statistical segmentation of streams of syllables: a pilot EEG study

Statistical segmentation of streams of syllables: a pilot EEG study

Is the distinction between intra- and extra-musical meaning implemented in the brain?: Comment on “Towards a neural basis of processing musical semantics” by Stefan Koelsch

Based on the results of a series of seminal studies very clearly reviewed in the paper, Stefan Koelsch proposes a neurobiological theory of musical meaning [4]. In this theory, the distinction between extraand intra-musical meaning [5] is implemented in the brain of the listener, with the N400 reflecting extra-musical meaning and the N5 associated with the processing of intra-musical meaning. The finding of an N400-like effect (unrelated minus related) in semantic priming experiments when the prime is a musical sound and the target a word (or vice versa) is indeed good evidence that the semantic properties of both elements are combined in an extra-musical conceptual space to determine whether they are semantically related or unrelated. Because the occurrence of words in an experimental design may encourage labeling of the musical sounds (e.g., mental generation of the word “heroic” when listening to an excerpt of Beethoven’s 3rd Symphony), one of the most interesting experimental condition is when two non-linguistic sounds are successively presented as prime and target. Importantly, Aramaki et al. [1], Frey et al. [2] and Grieser-Painter and Koelsch [3] reported an N400-like effect in such a condition of reduced linguistic mediation. However, if an N400 is generated in a purely musical condition (e.g., sound–sound) in which a musical element (be it a note, a chord, the first theme in the sonata form . . .) is driving expectations for another musical element, why should we consider the N400 as reflecting the processing of extra-musical meaning rather than the processing of intra-musical meaning? The possibility that the N400 also reflects intra-musical meaning is supported by the results of Steinbeis and Koelsch [6] showing an interaction between the N400 and the N5. This interaction implies that the processes underlying the N5 and N400 components are not independent and draw, at least partly, on the same pool of neural resources. It is thus possible that the N5 is, in fact, a delayed N400. In other words, the implementation in the brain of the extra-musical vs intra-musical meaning distinction may not be as straightforward as one would like it to

Neural Changes Associated with Nonspeech Auditory Category Learning Parallel Those of Speech Category Acquisition

Native language experience plays a critical role in shaping speech categorization, but the exact mechanisms by which it does so are not well understood. Investigating category learning of nonspeech sounds with which listeners have no prior experience allows their experience to be systematically controlled in a way that is impossible to achieve by studying natural speech acquisition, and it provides a means of probing the boundaries and constraints that general auditory perception and cognition bring to the task of speech category learning. In this study, we used a multimodal, video-game-based implicit learning paradigm to train participants to categorize acoustically complex, nonlinguistic sounds. MMN responses to the nonspeech stimuli were collected before and after training, and changes in MMN resulting from the nonspeech category learning closely resemble patterns of change typically observed during speech category learning. Results indicate that changes in mismatch negativity resulting from the nonspeech category learning closely resemble patterns of change typically observed during speech category learning. This suggests that the often-observed "specialized" neural responses to speech sounds may result, at least in part, from the expertise we develop with speech categories through experience rather than from properties unique to speech (e.g., linguistic or vocal tract gestural information). Furthermore, particular characteristics of the training paradigm may inform our understanding of mechanisms that support natural speech acquisition.

Acoustic processing of temporally modulated sounds in infants: evidence from a combined near-infrared spectroscopy and EEG study.

Speech perception requires rapid extraction of the linguistic content from the acoustic signal. The ability to efficiently process rapid changes in auditory information is important for decoding speech and thereby crucial during language acquisition. Investigating functional networks of speech perception in infancy might elucidate neuronal ensembles supporting perceptual abilities that gate language acquisition. Interhemispheric specializations for language have been demonstrated in infants. How these asymmetries are shaped by basic temporal acoustic properties is under debate. We recently provided evidence that newborns process non-linguistic sounds sharing temporal features with language in a differential and lateralized fashion. The present study used the same material while measuring brain responses of 6 and 3 month old infants using simultaneous recordings of electroencephalography (EEG) and near-infrared spectroscopy (NIRS). NIRS reveals that the lateralization observed in newborns remains constant over the first months of life. While fast acoustic modulations elicit bilateral neuronal activations, slow modulations lead to right-lateralized responses. Additionally, auditory-evoked potentials and oscillatory EEG responses show differential responses for fast and slow modulations indicating a sensitivity for temporal acoustic variations. Oscillatory responses reveal an effect of development, that is, 6 but not 3 month old infants show stronger theta-band desynchronization for slowly modulated sounds. Whether this developmental effect is due to increasing fine-grained perception for spectrotemporal sounds in general remains speculative. Our findings support the notion that a more general specialization for acoustic properties can be considered the basis for lateralization of speech perception. The results show that concurrent assessment of vascular based imaging and electrophysiological responses have great potential in the research on language acquisition.

Lower-Frequency Event-Related Desynchronization: A Signature of Late Mismatch Responses to Sounds, Which Is Reduced or Absent in Children with Specific Language Impairment

Poor discrimination of nonlinguistic sounds has been implicated in language-learning problems in children, but research evidence has been inconsistent. This study included 32 participants with specific language impairment (SLI) and 32 typically developing controls aged 7-16 years. Frequency discrimination thresholds were estimated in a task where participants had to distinguish a higher-frequency tone from a 1000 Hz tone. Neurophysiological responses were assessed in an oddball paradigm. Stimuli were either 1030 or 1200 Hz pure tones (deviants) presented in a series of standard 1000 Hz tones, or syllables (deviant [da] or [bi] in a series of standard /ba/). On the behavioral task, children (7- to 11-year-olds) had high thresholds, regardless of language status, but teenagers (12-16 years) with SLI had higher thresholds than their controls. Conventional analysis of electrophysiological responses showed no difference between groups for the mismatch negativity (MMN), but the late discriminative negativity (LDN) was reduced in amplitude for smaller deviants in participants with SLI. Time-frequency analysis revealed that, whereas the MMN reflected enhanced intertrial coherence in the theta frequency band, the LDN corresponded to a period of event-related desynchronization extending across a wide low-frequency band including delta, theta, and alpha. This manifested as a drop in power in those frequencies, which was marked in the controls but reduced or absent in children with SLI across all stimulus types. This provides compelling evidence for a low-level auditory perceptual impairment in SLI that affects a processing stage after initial detection of a sound change.

Sound Categorization and Conceptual Priming for Nonlinguistic and Linguistic Sounds

The aim of these experiments was to compare conceptual priming for linguistic and for a homogeneous class of nonlinguistic sounds, impact sounds, by using both behavioral (percentage errors and RTs) and electrophysiological measures (ERPs). Experiment 1 aimed at studying the neural basis of impact sound categorization by creating typical and ambiguous sounds from different material categories (wood, metal, and glass). Ambiguous sounds were associated with slower RTs and larger N280, smaller P350/P550 components, and larger negative slow wave than typical impact sounds. Thus, ambiguous sounds were more difficult to categorize than typical sounds. A category membership task was used in Experiment 2. Typical sounds were followed by sounds from the same or from a different category or by ambiguous sounds. Words were followed by words, pseudowords, or nonwords. Error rate was highest for ambiguous sounds and for pseudowords and both elicited larger N400-like components than same typical sounds and words. Moreover, both different typical sounds and nonwords elicited P300 components. These results are discussed in terms of similar conceptual priming effects for nonlinguistic and linguistic stimuli.

Cochlear Implant-Mediated Perception of Nonlinguistic Sounds

The overwhelming majority of test measures to assess cochlear implant (CI) candidacy, efficacy, and progress are based on speech perception. Nonlinguistic sounds, such as alerting and nonspeech human generated sounds, have received comparatively little attention, despite their central importance for daily living and environmental sound awareness. The purposes of this study were to develop and validate a beta test measure of nonlinguistic sound perception and to assess performance in CI users. A beta test of nonlinguistic sound perception, referred to as the NonLinguistic Sounds Test (NLST) was developed. The NLST consists of 50 sound tokens distributed over five categories (animal, human nonspeech, mechanical/alerting, nature, and musical instruments). Both closed-set (category identification) and open-set (token identification) nonlinguistic sound perceptions were examined. Twenty-two postlingually deafened CI users (mean age, 59.4 +/- 10 yrs) were evaluated using common speech perception test measures (Hearing In Noise test and Consonant-Nucleus-Consonant words) and the NLST following a pilot study in which nonlinguistic sound tokens used were selected or eliminated. The NLST was easily administered to all 22 CI subjects. An overall token identification score of 49 +/- 13.5% correct was obtained across all five categories. CI users were able to identify the correct category for 71 +/- 11.5% of tokens. A moderate correlation between speech perception and accuracy of nonlinguistic identification was found (r = 0.519, p = 0.016). The results suggest that nonlinguistic sounds are difficult for CI users to perceive. The categorization and identification scores suggest that sounds with harmonic structure or sounds with repetitive temporal structure are easier for CI users to perceive. A further developed clinical version of the NLST may be a useful clinical test to measure CI performance and progress, and perception of nonlinguistic sounds should receive greater attention during postimplant auditory rehabilitation.

Music Perception in Children With Cochlear Implants

AbstractAs a result of the widespread use of cochlear implants, individuals with profound hearing loss now are able to hear sounds ranging from a syllable to a symphony. This form of “electric hearing” has been remarkably successful in providing sound to the deaf population and at least 100,000 implantation procedures have been performed worldwide in more than 80 countries (Clark, 2008). Today, it is routine for post-lingual deafened individuals (one who lost their hearing after normal childhood language acquisition) to achieve high performance on language tests following implantation (Lalwani, Larky, Wareing, Kwast, & Schindler, 1998). Deaf children implanted at an early age with a CI usually develop excellent spoken language skills, with placement into mainstream educational schooling (Francis, Koch, Wyatt, & Niparko, 1999). The overwhelming emphasis on language perception in CI users has led to relative neglect of non-linguistic sound perception. Yet, the auditory world consists of many other sounds besides those of spoken language. Of all non-linguistic sounds, perception of music—particularly pitch and timbre—represents the greatest challenge for implant-mediated listening (Limb, 2006). High-level perception of music rarely is attained through conventional speech processing technology in adults or children. Recent technological advances, however, have increased the processing capabilities of modern CIs and hold great promise for music perception and quality of life for children with cochlear implants (Lassaletta et al., 2007).

Statistical learning of adjacent and nonadjacent dependencies among nonlinguistic sounds

Previous work has demonstrated that adults are capable of learning patterned relationships among adjacent syllables or tones in continuous sequences but not among nonadjacent syllables. However, adults are capable of learning patterned relationships among nonadjacent elements (segments or tones) if those elements are perceptually similar. The present study significantly broadens the scope of this previous work by demonstrating that adults are capable of encoding the same types of structure among unfamiliar nonlinguistic and nonmusical elements but only after much more extensive exposure. We presented participants with continuous streams of nonlinguistic noises and tested their ability to recognize patterned relationships. Participants learned the patterns among noises within adjacent groups but not within nonadjacent groups unless a perceptual similarity cue was added. This result provides evidence both that statistical learning mechanisms empower adults to extract structure from nonlinguistic and nonmusical elements and that perceptual similarity eases constraints on nonadjacent pattern learning. Supplemental materials for this article can be downloaded from pbr.psychonomic-journals.org/content/supplemental.

Expertise with Artificial Nonspeech Sounds Recruits Speech-Sensitive Cortical Regions

Regions of the human temporal lobe show greater activation for speech than for other sounds. These differences may reflect intrinsically specialized domain-specific adaptations for processing speech, or they may be driven by the significant expertise we have in listening to the speech signal. To test the expertise hypothesis, we used a video-game-based paradigm that tacitly trained listeners to categorize acoustically complex, artificial nonlinguistic sounds. Before and after training, we used functional MRI to measure how expertise with these sounds modulated temporal lobe activation. Participants' ability to explicitly categorize the nonspeech sounds predicted the change in pretraining to posttraining activation in speech-sensitive regions of the left posterior superior temporal sulcus, suggesting that emergent auditory expertise may help drive this functional regionalization. Thus, seemingly domain-specific patterns of neural activation in higher cortical regions may be driven in part by experience-based restructuring of high-dimensional perceptual space.

Pediatric Assessment of Nonlinguistic Sound Perception: A Look Back and a Look Ahead to a Pediatric Nonlinguistic Sounds Test

Abstract The overwhelming majority of test measures to assess adult and pediatric cochlear implant candidacy, efficacy, and progress are based on speech perception. Nonlinguistic sounds have received comparatively little attention, despite their central importance for incidental learning, daily living and environmental sound awareness. The purpose of this review is to: 1. Highlight the importance of nonlinguistic sound perception, 2. Discuss currently available pediatric nonlinguistic sound perception profiles and behavioral measures, and 3. Describe both the Nonlinguistic Sounds Test (NLST) for adults and adolescents as well as the pediatric Picture-identification Nonlinguistic sounds Inventory for Children (PicNIC).

An acoustic comparison of nonlinguistic sounds to sentences spoken in American English.

Acoustic characteristics of nonlinguistic (nonspeech) sounds (NLSs) were measured for duration and spectral variation, and compared to acoustic characteristics of spoken sentences (TIMIT database). The NLS, included samples produced by animal, human, mechanical, and natural sources. The acoustic comparison examined stoplike onsets, fricativelike intervals, vowel‐like intervals, and syllablelike variations in amplitude. The NLSs were identified by two groups of listeners: listeners with normal hearing and users of cochlear implants. Results of the listening tests have been reported previously by Inverso et al. (2007) and Inverso (2008). All of the NLSs were identified accurately by listeners with normal hearing, but not by the users of cochlear implants. The current analysis focuses on the ways in which the NLSs are similar to and other ways in which they are different from sentences spoken by a variety of talkers. It was found that speechlike variation in amplitude, both in terms of duration and event onset/offset, was a strong cue for listeners with cochlear implants; that is, NLSs that contained distinct events that were similar in duration and amplitude variation to syllables in speech were identified more accurately than ones that did not.

Linguistic Labels and Categorization in Infancy: Do Labels Facilitate or Hinder?

Although it is generally accepted that labels facilitate categorization in infancy, recent evidence suggests that infants and young children are more likely to process visual input when presented in isolation than when paired with nonlinguistic sounds or linguistic labels. These findings suggest that auditory input (when compared to a no-auditory baseline) may hinder rather than facilitate categorization. This study assessed 8-month-olds' (n = 191) and 12-month-olds' (n = 81) abilities to form categories when images were paired with nonlinguistic sounds, linguistic labels, and when presented in isolation. Overall, infants accumulated more looking when visual stimuli were accompanied by sounds or labels; however, infants were more likely to categorize when the visual images were presented without an auditory stimulus.

Acoustical analysis of nonlinguistic sounds

The perception of speech is an undisputed important goal for cochlear implantation; however, the reception of nonlinguistic sounds (NLS) is also important. Nonlinguistic sounds are important for one’s safety and they are considered to have importance to a person’s sense of connection to and welfare in one’s environment. NLS are different from speech, and the perception and acoustic characteristics of NLS have not been studied adequately to allow clinicians to fit CIs for optimal recognition of both speech and NLS. The specific categories of nonlinguistic sounds to be evaluated in this study are (1) human vocal/nonverbal, such as a baby crying or a person coughing; (2) mechanical/alerting, such as a telephone or an alarm clock buzzing; (3) nature/inanimate, such as weather sounds; (4) animal/insect, such as a dog barking; and (5) musical instruments such as the strum of a guitar. In this ongoing study, 50 listeners (half adults with normal hearing and half adult post‐ling deafened CI users) are being evaluated with respect to their ability to identify the sources of a variety of common, everyday NLS. In this poster, an acoustic analysis of a representative set of the stimuli will be presented.

Selective attention to human voice enhances brain activity bilaterally in the superior temporal sulcus

Regional cerebral blood flow was measured with positron emission tomography (PET) in 10 healthy male volunteers. They heard two binaurally delivered concurrent stories, one spoken by a male voice and the other by a female voice. A third story was presented at the same time as a text running on a screen. The subjects were instructed to attend silently to one of the stories at a time. In an additional resting condition, no stories were delivered. PET data showed that in comparison with the reading condition, the brain activity in the speech–listening conditions was enhanced bilaterally in the anterior superior temporal sulcus including cortical areas that have been reported to be specifically sensitive to human voice. Previous studies on attention to non-linguistic sounds and visual objects, in turn, showed prefrontal activations that are presumably related to attentional control functions. However, comparisons of the present speech–listening and reading conditions with each other or with the resting condition indicated no prefrontal activity, except for an activation in the inferior frontal cortex that was presumably associated with semantic and syntactic processing of the attended story. Thus, speech listening, as well as reading, even in a distracting environment appears to depend less on the prefrontal control functions than do other types of attention-demanding tasks, probably because selective attention to speech and written text are over-learned actions rehearsed daily.

Effects of later-occurring nonlinguistic sounds on speech categorization

Nonspeech stimuli influence phonetic categorization, but effects observed so far have been limited to precursors' influence on perception of following speech. However, both preceding and following speech affect phonetic categorization. This asymmetry raises questions about whether general auditory processes play a role in context-dependent speech perception. This study tested whether the asymmetry stems from methodological issues or genuine mechanistic limitations. To determine whether and how backward effects of nonspeech context on speech may occur, one experiment examined perception of CVC words with [ga]-[da] series onsets followed by one of two possible embedded tones and one of two possible final consonants. When the tone was separated from the target onset by 100 ms, contrastive effects of tone frequency similar to those of previous studies were observed; however, when the tone was moved closer to the target segment assimilative effects were observed. In another experiment, contrastive effects of a following tone were observed in both CVC words and CV nonwords, although the size of the effects depended on syllable structure. Results are discussed with respect to contrastive mechanisms not speech-specific but operating at a relatively high level, taking into account spectrotemporal patterns occurring over extended periods before and after target events.

When space merges into language

We present data from right brain-damaged patients, with and without spatial heminattention, which show the influence of hemispatial deficits on spoken language processing. We explored the findings of a previous study, which used an emphatic stress detection task and suggested spatial transcoding of a spoken active sentence in a ‘language line’. This transcoding was impaired in its initial portion (the subject-word) when the neglect syndrome was present. By expanding the original methodology, the present study provides a deeper understanding of the level of spoken language processing involved in the heminattentional bias. To ascertain the role played by syntactic structure, active and passive sentences were compared. Sentences comprised of musical notes and of a sequence of unrelated nouns were also compared to determine whether the bias was manifest with any sequence of events (not only linguistic ones) deployed over time, and with a sequence of linguistic events not embedded in a structured syntactic frame. Results showed that heminattention exerted an influence only when a syntactically structured linguistic input (=sentence with agent of action, action and recipient of action) was processed, and that it did not interfere when a sequence of non-linguistic sounds or unrelated words was presented. Furthermore, when passing from active to passive sentences, the heminattentional bias was inverted, suggesting that heminattention primarily involves the logical subject of the sentence, which has an inverted position in passive sentences. These results strongly suggest that heminattention acts on the spatial transcoding of the deep structure of spoken language.

It's a scream: professional hearing and tape fetishism

Abstract This paper addresses the roles of taping and tapes in the arenas of academic Conversation and Discourse Analysis, and in a recent American trial of therapists which constituted a major development in the Recovered Memory/False Memory debate. Our argument is that two seemingly opposed features of the practice of hearing tapes—tape fetishism and professional hearing—are in fact interdependent. By tape fetishism we mean the treatment of the tape as a direct and evidential record of a past event, and thus as a quasi-magical time machine. Professional hearing is a trained method of hearing—as developed, for example, in conversation analysis. The joint operation of these features prevents us from seeing that all hearings are mediated, and that their reports are interpretative. The paper sets out to analyze modes of mediation: the analytic glossing of voiced but non-linguistic sounds (laughing, crying, screaming) and the use of rhetorical descriptions in media reports of taped sounds.