Unrelated Sounds Research Articles

Relationships among song characteristics in Japanese tits

Birdsongs often contain two simultaneous, harmonically unrelated sounds; however, little is known of the function of this two-voice phenomenon. We hypothesized that it contributes to increases in song speed and note repertoire, which are important song characteristics in sexual selection. We analyzed six song variables in Japanese tits (Parus minor) and found positive correlations between all possible pairs of the proportion in length of two-voice vocalization, note rate, and note type repertoire, which is consistent with our hypothesis. This finding is a first step towards determining the function of two-voice vocalization in song communication.

Lip-Reading Enables the Brain to Synthesize Auditory Features of Unknown Silent Speech.

Lip-reading is crucial for understanding speech in challenging conditions. But how the brain extracts meaning from, silent, visual speech is still under debate. Lip-reading in silence activates the auditory cortices, but it is not known whether such activation reflects immediate synthesis of the corresponding auditory stimulus or imagery of unrelated sounds. To disentangle these possibilities, we used magnetoencephalography to evaluate how cortical activity in 28 healthy adult humans (17 females) entrained to the auditory speech envelope and lip movements (mouth opening) when listening to a spoken story without visual input (audio-only), and when seeing a silent video of a speaker articulating another story (video-only). In video-only, auditory cortical activity entrained to the absent auditory signal at frequencies <1 Hz more than to the seen lip movements. This entrainment process was characterized by an auditory-speech-to-brain delay of ∼70 ms in the left hemisphere, compared with ∼20 ms in audio-only. Entrainment to mouth opening was found in the right angular gyrus at <1 Hz, and in early visual cortices at 1-8 Hz. These findings demonstrate that the brain can use a silent lip-read signal to synthesize a coarse-grained auditory speech representation in early auditory cortices. Our data indicate the following underlying oscillatory mechanism: seeing lip movements first modulates neuronal activity in early visual cortices at frequencies that match articulatory lip movements; the right angular gyrus then extracts slower features of lip movements, mapping them onto the corresponding speech sound features; this information is fed to auditory cortices, most likely facilitating speech parsing.SIGNIFICANCE STATEMENT Lip-reading consists in decoding speech based on visual information derived from observation of a speaker's articulatory facial gestures. Lip-reading is known to improve auditory speech understanding, especially when speech is degraded. Interestingly, lip-reading in silence still activates the auditory cortices, even when participants do not know what the absent auditory signal should be. However, it was uncertain what such activation reflected. Here, using magnetoencephalographic recordings, we demonstrate that it reflects fast synthesis of the auditory stimulus rather than mental imagery of unrelated, speech or non-speech, sounds. Our results also shed light on the oscillatory dynamics underlying lip-reading.

An Emergent Discriminative Learning Is Elicited During Multifrequency Testing.

In auditory-conditioned fear learning, the freezing response is independent of the sound frequencies used, but the frequency of the conditioned sound is considered distinct from those of unrelated sounds based on electrophysiological responses in the auditory system. Whether an emergent discriminative learning underlies auditory fear conditioning and which nuclei and pathways are involved in it remain unclear. Using behavioral and electrophysiological assays, we found that the response of medial prefrontal cortex (mPFC) neurons to a conditioned auditory stimulus (CS) was enhanced relative to the response to unrelated frequencies (UFs) after auditory fear conditioning, and mice could distinguish the CS during multifrequency testing, a phenomenon called emergent discriminative learning. After silencing the mPFC with muscimol, emergent discriminative learning was blocked. In addition, the pure tone responses of mPFC neurons were inhibited after injection of lidocaine in the ipsilateral primary auditory cortex (A1), and the emergent discriminative learning was blocked by silencing both sides of A1 with muscimol. This study, therefore, provides evidence for an emergent discriminative learning mediated by mPFC and A1 neurons after auditory fear conditioning.

Neighing, barking, and drumming horses-object related sounds help and hinder picture naming.

The study presented here investigated how environmental sounds influence picture naming. In a series of four experiments participants named pictures (e.g., the picture of a horse) while hearing task-irrelevant sounds (e.g., neighing, barking, or drumming). Experiments 1 and 2 established two findings, facilitation from congruent sounds (e.g., picture: horse, sound: neighing) and interference from semantically related sounds (e.g., sound: barking), both relative to unrelated sounds (e.g., sound: drumming). Experiment 3 replicated the effects in a situation in which participants were not familiarized with the sounds prior to the experiment. Experiment 4 replicated the congruency facilitation effect, but showed that semantic interference was not obtained with distractor sounds which were not associated with target pictures (i.e., were not part of the response set). The general pattern of facilitation from congruent sound distractors and interference from semantically related sound distractors resembles the pattern commonly observed with distractor words. This parallelism suggests that the underlying processes are not specific to either distractor words or distractor sounds but instead reflect general aspects of semantic-lexical selection in language production. The results indicate that language production theories need to include a competitive selection mechanism at either the lexical processing stage, or the prelexical processing stage, or both. (PsycINFO Database Record

Influence of auditory spatial attention on cross-modal semantic priming effect: evidence from N400 effect.

Semantic priming is an important research topic in the field of cognitive neuroscience. Previous studies have shown that the uni-modal semantic priming effect can be modulated by attention. However, the influence of attention on cross-modal semantic priming is unclear. To investigate this issue, the present study combined a cross-modal semantic priming paradigm with an auditory spatial attention paradigm, presenting the visual pictures as the prime stimuli and the semantically related or unrelated sounds as the target stimuli. Event-related potentials results showed that when the target sound was attended to, the N400 effect was evoked. The N400 effect was also observed when the target sound was not attended to, demonstrating that the cross-modal semantic priming effect persists even though the target stimulus is not focused on. Further analyses revealed that the N400 effect evoked by the unattended sound was significantly lower than the effect evoked by the attended sound. This contrast provides new evidence that the cross-modal semantic priming effect can be modulated by attention.

Multiple sound source localization when sounds are stationary or rotating

In the field of spatial hearing, it is known that the sound source localization error increases when distractors are present. This study first determined the maximum number of sound sources that can be perceived. In one experiment, listeners determined how many sources (loudspeakers) presented sounds and these loudspeaker locations. In another experiment, listeners were asked to locate a new sound source which was added to an existing set of fixed sources. Listeners determined up to about four sources producing simultaneous unrelated speech sounds in both experiments. A second part of the research examined the effect of rotating the sound sources on the listener’s perception. If multiple unrelated sounds such as speech rotated, listeners could much more easily determine sound rotation than if the sounds were related to each other. If the sounds were related (e.g., all sounds were harmonics of a fundamental) but were still presented from a number of different loudspeakers, listeners could hardly process the individual sounds even when sounds rotated among the loudspeakers. Thus, the ability to locate multiple sound sources depends on the total number and motion of sources and the type of sound. [Research supported by an AFOSR grant.]

Music and Language Expertise Influence the Categorization of Speech and Musical Sounds: Behavioral and Electrophysiological Measurements

In this study, we used high-density EEG to evaluate whether speech and music expertise has an influence on the categorization of expertise-related and unrelated sounds. With this purpose in mind, we compared the categorization of speech, music, and neutral sounds between professional musicians, simultaneous interpreters (SIs), and controls in response to morphed speech-noise, music-noise, and speech-music continua. Our hypothesis was that music and language expertise will strengthen the memory representations of prototypical sounds, which act as a perceptual magnet for morphed variants. This means that the prototype would "attract" variants. This so-called magnet effect should be manifested by an increased assignment of morphed items to the trained category, by a reduced maximal slope of the psychometric function, as well as by differential event-related brain responses reflecting memory comparison processes (i.e., N400 and P600 responses). As a main result, we provide first evidence for a domain-specific behavioral bias of musicians and SIs toward the trained categories, namely music and speech. In addition, SIs showed a bias toward musical items, indicating that interpreting training has a generic influence on the cognitive representation of spectrotemporal signals with similar acoustic properties to speech sounds. Notably, EEG measurements revealed clear distinct N400 and P600 responses to both prototypical and ambiguous items between the three groups at anterior, central, and posterior scalp sites. These differential N400 and P600 responses represent synchronous activity occurring across widely distributed brain networks, and indicate a dynamical recruitment of memory processes that vary as a function of training and expertise.

Planning-related motor processes underlie mental practice and imitation learning.

It is still controversial whether mental practice-the internal rehearsal of movements to improve later performance-relies on processes engaged during physical motor performance and, if so, which processes these are. We report data from 5 experiments, in which participants mentally practiced complex rhythms with either feet or hands while using the same or different body parts to respond to unrelated sounds. We found that responses were impaired for those body parts that were concurrently used in mental practice, suggesting a binding of body-part-specific motor processes to action plans. This result was found when participants mentally trained to memorize the rhythms, to merely improve their performance, when mental practice and execution directly followed one another and when separated by a different task. Finally, it was found irrespective of whether participants practiced on the basis of a symbolic rhythm description and when they practiced by watching somebody perform the rhythms (imitation learning). The effect was eliminated only when the requirement for mental practice was eliminated from the task while keeping visual stimulation identical. These data link mental practice not to execution but planning related motor processes and reveal that these planning processes underlie both mental practice and imitation learning.

Product-related sounds speed visual search

Searching for a particular product in a supermarket can be a challenging business. The question therefore arises as to whether cues from the shopper’s other senses can be used to facilitate, guide, or bias visual search toward a particular product or product type. Prior research suggests that characteristic sounds can facilitate visual object localization (Iordanescu et al., 2008, 2010). Extending these findings to an applied setting, we investigated whether product-related sounds would facilitate visual search for products from different categories (e.g., champagne, potato crisps, deodorant) when arranged on a virtual shelf. On each trial, participants were visually presented with the name of a target product and then located the target within a virtual shelf display containing pictures of four different products (randomly selected from a set of nine). The visual display was randomly accompanied by a target-congruent, a target-incongruent, an unrelated, or no sound. Congruent sounds were semantically related to the target (e.g., uncorking a champagne bottle), incongruent sounds were related to the product shown in the corner opposite to the target, and unrelated sounds did not correspond to any of the products shown in the display. Participants found the target product significantly faster when the sound was congruent rather than incongruent with the target. All other pairwise comparisons were non-significant. These results extend the facilitatory crossmodal effect of characteristic sounds on visual search performance described earlier to the more realistic context of a virtual shelf display, showing that characteristic sounds can crossmodally enhance the visual processing of actual products.

Auditory Scene Analysis and the Role of Phenomenology in Experimental Psychology.

Abstract This paper is about two topics: a) the role of subjectivity in psychological research; and, b) my research on the perceptual organization of sound, in which subjectivity has played an important role. Audio demonstrations that appeal to the subjective experience of the reader are presented in lieu of objective research data to support the claims made about auditory organization. It is argued that all psychological research depends on an underlying framework of intuition (unformalized knowledge acquired through everyday experience), and that this intuition plays a role in the design of the experiment. Personal Experience as Data The personal experience of the researcher has not fared well in scientific psychology. Since the failure of Titchener's Introspectionism in the early 20th century, and the rise of Behaviourism, scientific psychology has harboured a deep suspicion of the experience of the researcher as an acceptable tool in research. One would think that the study of perception would be exempt from this suspicion, since the subject matter of the psychology of perception is supposed to be about how a person's experience is derived from sensory input. Instead, academic psychology, in its behaviouristic zeal, redefined perception as the ability to respond differently to different stimuli - bringing it into the stimulus-response framework. Despite Behaviourism's fall from grace, psychology still insists on a behaviouristic research methodology. In my own research, however, subjectivity has played a central role. It was a perceptual experience that got me going on the topic of perceptual organization in the first place: I was preparing an experiment on learning, involving a rapid sequence of unrelated sounds, each about the length of a speech phoneme. I spliced together one-tenth-second segments of many different sounds - water splashing in a sink, a dentist's drill, a tone, a vowel, etc. When I played the tape back to myself, though, I did not experience the sequences in the order that they were recorded on the tape. It appeared that nonadjacent sounds were grouping together and appeared to be adjacent. It was the similar sounds that seemed to be forming integrated perceptual sequences. This reminded me of an essay I had written at the University of Toronto on the topic of Gestalt Psychology. Some of the Gestaltist examples showed that similar visual forms would group together and segregate from dissimilar ones. Perhaps an analogous sort of grouping might be happening in my auditory sequence. Although I had never been trained in auditory perception research, this one subjective experience set me off on a 36-year period of study. When I use the term phenomenology, it is not with the technical meaning it has in the writings of Husserl or Heidegger, but is just a fancy name for experience. In all my years of research on the conditions under which a mixture of sounds will blend or be heard as separate sounds, my own phenomenology has played a central role in deciding what to study and how to study it. Also, the subjective experiences of colleagues and students have made it possible for them to understand the phenomena by listening to auditory demonstrations. I almost never carried out a study whose outcome I did not know in advance by listening to the stimuli. Only when I had figured out the conditions that would give rise to the effect I wanted to study, would I design a formal experiment. It is impossible to overestimate how many years of research this saved. It made it possible to study a large set of theoretical issues in perceptual organization without elaborate sets of preliminary experiments to establish the right parameters. The approach of listening to many variants of the signals and getting familiar with their effects at a personal level permitted us to speed up the development of a general overview of auditory grouping (Bregman, 1990/1994), rather than to merely accumulate more and more highly quantitative knowledge about a narrow experimental paradigm - a not-unknown practice in experimental psychology. …

Using spatialized sound cues in an auditorily rich environment

Previous Navy research has demonstrated that spatialized sound cues in an otherwise quiet setting are useful for directing attention and improving performance by 16.8% or more in the decision component of a complex dual-task. To examine whether the benefits of this technique are undermined in the presence of additional, unrelated sounds, a background recording of operations in a Navy command center and a voice communications response task [Bolia et al., J. Acoust. Soc. Am. 107, 1065–1066 (2000)] were used to simulate the conditions of an auditorily rich military environment. Without the benefit of spatialized sound cues, performance in the presence of this extraneous auditory information, as measured by decision response times, was an average of 13.6% worse than baseline performance in an earlier study. Performance improved when the cues were present by an average of 18.3%, but this improvement remained below the improvement observed in the baseline study by an average of 11.5%. It is concluded that while the two types of extraneous sound information used in this study degrade performance in the decision task, there is no interaction with the relative performance benefit provided by the use of spatialized auditory cues. [Work supported by ONR.]

Conceptual relationships between spoken words and environmental sounds: Event-related brain potential measures

Identifiable nonspeech sounds were paired with spoken words. In Experiment 1, words preceded by related sounds yielded faster lexical decision times than those preceded by unrelated sounds. In Experiment 2, subjects were presented with sound/word and word/sound pairs while event-related potentials were recorded. Words preceded by related sounds elicited smaller N400 components than those preceded by unrelated sounds; this N400 context effect was slightly larger at electrode sites over the right hemisphere. A context effect similar in latency and morphology was observed for sounds—a smaller negative wave for related than unrelated sounds. The context effect for sounds was significantly larger at left than right recording sites, suggesting differential hemispheric involvement in the processing of word meanings than the “meanings” of environmental sounds.

Temporal discrimination of recycled tonal sequences: Pattern matching and naming of order by untrained listeners

Sets of recycled sequences of four successive tones were presented in all six possible orders to untrained listeners. For pitches within the musical range, recognition (as measured by matching of any unknown order with an array of permuted orders of the same tones) could be accomplished as readily for tonal durations and frequency separations outside the limits employed for melodic construction as inside these limits. Identifying or naming of relative pitches of successive tones was considerably more difficult than matching for these tonal sequences, and appeared to follow different rules based upon duration and upon frequency separation. Use of frequencies above the pitch limits for music (4,500 Hz and above) resulted in poor performance both for matching and naming of order. Introduction of short silent intervals between items was without effect for both tasks. Naming of order and pattern recognition appear to reflect different basic processes, in agreement with earlier formulations based on experiments with phonemic sequences of speech and sequences of unrelated sounds (hisses, tones, buzzes). Special characteristics of tonal sequences are discussed, and some speculations concerning music are offered.

Auditory pattern recognition by untrained listeners

Earlier work with unpracticed Ss has indicated that identification (naming) of the temporal order of components within repeated sequences consisting of three or four unrelated sounds cannot be accomplished when the item durations are 200 msec or less. In the present experiment, separate groups of 30 unpracticed Ss were required to teil whether alternated sequences, each consisting of reiterated presentations of the same three or four successive items, were in identical or permuted order. Naming of the order within the sequences was not required. Accuracy of same/different judgments was significantly better than chance when all items lasted 200 msec. Changing the duration of each item in one of the two sequences above or below 200 msec made the task more difficult. These results, together with other evidence, suggest that: (1) identification of order and recognition of auditory temporal patterns may represent fundamentally different processes, and (2) recognition may involve matching of “temporal templates.”

Identification of temporal order within auditory sequences

Unpracticed Ss reported the order of sounds in sequences consisting of either three or four successive items repeated over and over without pause. With unrelated sounds each lasting 200 msec, correct reports of order were at chance level for oral responses and for card-ordering responses (each card bearing the name of one sound). The sequences with four unrelated items were studied in greater detail, and the threshold for identification was found to be 670 msec with oral responses and 300 msec with card-ordering responses. When two related sounds (tones) were used in four-item sequences, correct card-ordering was possible at 200 msec per item when the tones were temporally contiguous, but was not possible at this duration when the tones were separated by nonrelated sounds. Some special rules governing auditory sequence identification were suggested, and implications for theories of auditory perception discussed.