Auditory Categorization Research Articles

Cross-modal transfer after auditory task-switching training

Task-switching training was shown to improve performance not only for the trained tasks (i.e., reduced performance costs resulting from the task switches), but also for structurally similar (near transfer) or even dissimilar tasks (far transfer). However, it is still unclear whether the improvement is specific to the trained input modality or whether cognitive control occurs at an amodal processing level enabling transfer of set-shifting abilities to different input modalities. In this study, training and transfer was assessed for an auditory task-switching paradigm in which spoken words from different semantic categories were presented dichotically requiring participants to switch between two auditory categorization tasks. Cross-modal transfer of task-switching training was assessed in terms of the performance costs in a visual task-switching situation using tasks that were structurally similar to the trained tasks. The 4-day training significantly reduced the costs resulting from mixing the two auditory tasks, as compared to both an active (auditory single-task training) and a passive control group (no training). More importantly, the auditory task-switching training was also found to reduce the mixing costs for untrained visual tasks, indicating cross-modal transfer. This finding suggests that the improvement resulting from task-switching training is not specific to the trained stimulus modality, but it seems to be driven by a cognitive control mechanism operating at an amodal processing level. The training did not reveal any far-transfer effects to working memory, inhibition, or fluid intelligence, suggesting that the modality-independent enhancement of set-shifting does not generalize to other cognitive control functions.

Perceptual dimensions influence auditory category learning.

Human category learning appears to be supported by dual learning systems. Previous research indicates the engagement of distinct neural systems in learning categories that require selective attention to dimensions versus those that require integration across dimensions. This evidence has largely come from studies of learning across perceptually separable visual dimensions, but recent research has applied dual system models to understanding auditory and speech categorization. Since differential engagement of the dual learning systems is closely related to selective attention to input dimensions, it may be important that acoustic dimensions are quite often perceptually integral and difficult to attend to selectively. We investigated this issue across artificial auditory categories defined by center frequency and modulation frequency acoustic dimensions. Learners demonstrated a bias to integrate across the dimensions, rather than to selectively attend, and the bias specifically reflected a positive correlation between the dimensions. Further, we found that the acoustic dimensions did not equivalently contribute to categorization decisions. These results demonstrate the need to reconsider the assumption that the orthogonal input dimensions used in designing an experiment are indeed orthogonal in perceptual space as there are important implications for category learning.

The Impact of Using Modern Information and Communication Equipment and Services on Driving Safety

With the development and application of modern information and communication equipment and services (ICES), global society has undergone significant development and progress in everyday life. Along with the positive, negative aspects of their usage have evolved. One of the problems is the new sources of driver distraction, which ultimately results in a serious, growing daily threat to traffic safety. Simultaneously with the development of ICES, the number of distracters (mobile terminal devices, laptops, smart watches, and others) has increased as well, which also reduces the level of safety and affects the quality of driving. With the increase in participation and ownership of mobile terminal devices, as well as the implementation of modern multimedia information and communication systems in vehicles, this problem will further escalate in the coming years. The unfavorable factor comes from the fact that distraction factors influenced by using ICES belong to bio-mechanical, cognitive, auditory, and visual categories, which directly affects essential human functions for successful and safe vehicle operation. This paper provides insight into the issue of distraction factors that affect drivers with an emphasis on those caused by using modern ICES.

Decoding sound categories based on whole-brain functional connectivity patterns.

2Sound decoding is important for patients with sensory loss, such as the blind. Previous studies on sound categorization were conducted by estimating brain activity using univariate analysis or voxel-wise multivariate decoding methods and suggested some regions were sensitive to auditory categories. It is proposed that feedback connections between brain areas may facilitate auditory object selection. Therefore, it is important to explore whether functional connectivity among regions can be used to decode sound category. In this study, we constructed whole-brain functional connectivity patterns when subjects perceived four different sound categories and combined them with multivariate pattern classification analysis for sound decoding. The categorical discriminative networks and regions were determined based on the weight maps. Results showed that a high accuracy in multi-category classification was obtained based on the whole-brain functional connectivity patterns and the results were verified by different preprocessing parameters. Insight into the category discriminative functional networks showed that contributive connections crossed the left and right brain, and ranged from primary regions to high-level cognitive regions, which provide new evidence for the distributed representation of auditory object. Further analysis of brain regions in the discriminative networks showed that superior temporal gyrus and Heschl's gyrus significantly contributed to discriminating sound categories. Together, the findings reveal that functional connectivity based multivariate classification method provides rich information for auditory category decoding. The successful decoding results implicate the interactive properties of the distributed brain areas in auditory sound representation.

Distributional Information Shapes Infants’ Categorization of Objects

While distributional learning has been successfully demonstrated for auditory categorization, this study tests whether this mechanism also applies to object categorization: Ten‐month‐olds (n = 38) were familiarized with either a unimodal or bimodal distribution of a visual continuum. Using automatic eye tracking, we assessed categorization through the alternating/nonalternating paradigm. For infants in the bimodal condition, their average dwell time was larger for alternating trials than for nonalternating trials, while infants in the unimodal condition initially looked equally long at both types of trials. This group difference suggests that the shape of frequency distribution bears on the number of categories that infants construct from a continuum. Later in test, all infants show this alternating preference. We conclude that categorization is a flexible process, continuously adjusting itself to additional input.

Acquisition of categorical perception of Mandarin tone

This experiment examined the effect of motor repetition on the learning of Mandarin pitch categories by non-native speakers. In Mandarin, there are four tonal categories, which refer to pitch contours that discriminate words. Differentiating tonal categories is both essential and arduous for non-native speakers to learn. Native speakers do this effortlessly because they have a categorical perception (CP) effect for tones, i.e. they perceive items within a tonal category as more similar to each other and items between categories as more different. Non-native speakers do not have this effect, and this experiment attempted to induce the CP effect for Mandarin tones via a training task, which was the experimental manipulation: Participants either repeated a tone stimulus before categorizing it or listened to a stimulus and categorized it without repetition. Discrimination between and within tonal categories was measured before and after training. All participants demonstrated increased between-category and within-category discrimination after training, except for learners who repeated stimuli in the training phase. They demonstrated a decrease in within-category discrimination, showing a weak CP effect that could be stronger with more training. Implications of these results on auditory category learning and language education will be discussed.

The balanced act of crossmodal and intramodal plasticity: Enhanced representation of auditory categories in the occipital cortex of early blind people links to reduced temporal coding

The balanced act of crossmodal and intramodal plasticity: Enhanced representation of auditory categories in the occipital cortex of early blind people links to reduced temporal coding

The Contribution of Primary Auditory Cortex to Auditory Categorization in Behaving Monkeys.

The specific contribution of core auditory cortex to auditory perception –such as categorization– remains controversial. To identify a contribution of the primary auditory cortex (A1) to perception, we recorded A1 activity while monkeys reported whether a temporal sequence of tone bursts was heard as having a “small” or “large” frequency difference. We found that A1 had frequency-tuned responses that habituated, independent of frequency content, as this auditory sequence unfolded over time. We also found that A1 firing rate was modulated by the monkeys’ reports of “small” and “large” frequency differences; this modulation correlated with their behavioral performance. These findings are consistent with the hypothesis that A1 contributes to the processes underlying auditory categorization.

Task and distribution sampling affect auditory category learning.

There is substantial evidence that two distinct learning systems are engaged in category learning. One is principally engaged when learning requires selective attention to a single dimension (rule-based), and the other is drawn online by categories requiring integration across two or more dimensions (information-integration). This distinction has largely been drawn from studies of visual categories learned via overt category decisions and explicit feedback. Recent research has extended this model to auditory categories, the nature of which introduces new questions for research. With the present experiment, we addressed the influences of incidental versus overt training and category distribution sampling on learning information-integration and rule-based auditory categories. The results demonstrate that the training task influences category learning, with overt feedback generally outperforming incidental feedback. Additionally, distribution sampling (probabilistic or deterministic) and category type (information-integration or rule-based) both affect how well participants are able to learn. Specifically, rule-based categories are learned equivalently, regardless of distribution sampling, whereas information-integration categories are learned better with deterministic than with probabilistic sampling. The interactions of distribution sampling, category type, and kind of feedback impacted category-learning performance, but these interactions have not yet been integrated into existing category-learning models. These results suggest new dimensions for understanding category learning, inspired by the real-world properties of auditory categories.

A Computational Account of the Role of Cochlear Nucleus and Inferior Colliculus in Stabilizing Auditory Nerve Firing for Auditory Category Learning.

It is well known that auditory nerve (AN) fibers overcome bandwidth limitations through the volley principle, a form of multiplexing. What is less well known is that the volley principle introduces a degree of unpredictability into AN neural firing patterns that may be affecting even simple stimulus categorization learning. We use a physiologically grounded, unsupervised spiking neural network model of the auditory brain with spike time dependent plasticity learning to demonstrate that plastic auditory cortex is unable to learn even simple auditory object categories when exposed to the raw AN firing input without subcortical preprocessing. We then demonstrate the importance of nonplastic subcortical preprocessing within the cochlear nucleus and the inferior colliculus for stabilizing and denoising AN responses. Such preprocessing enables the plastic auditory cortex to learn efficient robust representations of the auditory object categories. The biological realism of our model makes it suitable for generating neurophysiologically testable hypotheses.

Using a Digital Language Processor to Quantify the Auditory Environment and the Effect of Hearing Aids for Adults with Hearing Loss

Auditory environments can influence the communication function of individuals with hearing loss and the effects of hearing aids. Therefore, a tool that can objectively characterize a patient's real-world auditory environments is needed. To use the Language Environment Analysis (LENA) system to quantify the auditory environments of adults with hearing loss, to examine if the use of hearing aids changes a user's auditory environment, and to determine the association between LENA variables and self-report hearing aid outcome measures. This study used a crossover design. Participants included 22 adults with mild-to-moderate hearing loss, age 64-82 yr. Participants were fitted with bilateral behind-the-ear hearing aids from a major manufacturer. The LENA system consists of a digital language processor (DLP) that is worn by an individual and records up to 16 hr of the individual's auditory environment. The recording is then automatically categorized according to time spent in different types of auditory environments (e.g., meaningful speech and TV/electronic sound) by the LENA algorithms. The LENA system also characterizes the user's auditory environment by providing the sound levels of different auditory categories. Participants in the present study wore a LENA DLP in an unaided condition and aided condition, which each lasted six to eight days. Participants wore bilateral hearing aids in the aided condition. Percentage of time spent in each auditory environment, as well as median levels of TV/electronic sounds and speech, were compared between subjects' unaided and aided conditions using paired sample t tests. LENA data were also compared to self-report measures of hearing disability and hearing aid benefit using Pearson correlations. Overall, participants spent the greatest percentage of time in silence (∼40%), relative to other auditory environments. Participants spent ∼12% and 26% of their time in meaningful speech and TV/electronic sound environments, respectively. No significant differences were found between mean percentage of time spent in each auditory environment in the unaided and aided conditions. Median TV/electronic sound levels were on average 2.4 dB lower in the aided condition than in the unaided condition; speech levels were not significantly different between the two conditions. TV/electronic sound and speech levels did not significantly correlate with self-report data. The LENA system can provide rich data to characterize the everyday auditory environments of older adults with hearing loss. Although TV/electronic sound level was significantly lower in the aided than unaided condition, the use of hearing aids seemed not to substantially change users' auditory environments. Because there is no significant association between objective LENA variables and self-report questionnaire outcomes, these two types of measures may assess different aspects of communication function. The feasibility of using LENA in clinical settings is discussed.

Training Humans to Categorize Monkey Calls: Auditory Feature- and Category-Selective Neural Tuning Changes

Grouping auditory stimuli into common categories is essential for a variety of auditory tasks, including speech recognition. We trained human participants to categorize auditory stimuli from a large novel set of morphed monkey vocalizations. Using fMRI-rapid adaptation (fMRI-RA) and multi-voxel pattern analysis (MVPA) techniques, we gained evidence that categorization training results in two distinct sets of changes: sharpened tuning to monkey call features (without explicit category representation) in left auditory cortex and category selectivity for different types of calls in lateral prefrontal cortex. In addition, the sharpness of neural selectivity in left auditory cortex, as estimated with both fMRI-RA and MVPA, predicted the steepness of the categorical boundary,whereas categorical judgment correlated with release from adaptation in the left inferior frontal gyrus. These results support the theory that auditory category learning follows a two-stage model analogous to the visual domain, suggesting general principles of perceptual category learning in the human brain.

FMRI Decoding of Auditory Information Based on Multi-voxel Pattern Analysis

Auditory information decoding is very important for human-machine interaction and brain-inspired study. Whether the auditory content and location information can be both decoded from the human brain is still unclear. In this study, we employed support vector machine (SVM) classification method to decoding the auditory category and direction based on brain signals in two auditory core regions (STG and BA 41, 42 regions). Results showed that the proposed method successfully decoded the categories and directions with high accuracies than chance level (p<0.05). The results also suggested the totally blind subjects are better at discriminating the auditory categories and directions in STG. The findings in this study demonstrate the effectiveness of SVM in brain auditory information decoding and also promote our understanding of the auditory cognitive mechanism.

Varying acoustic-phonemic ambiguity reveals that talker normalization is obligatory in speech processing.

The nondeterministic relationship between speech acoustics and abstract phonemic representations imposes a challenge for listeners to maintain perceptual constancy despite the highly variable acoustic realization of speech. Talker normalization facilitates speech processing by reducing the degrees of freedom for mapping between encountered speech and phonemic representations. While this process has been proposed to facilitate the perception of ambiguous speech sounds, it is currently unknown whether talker normalization is affected by the degree of potential ambiguity in acoustic-phonemic mapping. We explored the effects of talker normalization on speech processing in a series of speeded classification paradigms, parametrically manipulating the potential for inconsistent acoustic-phonemic relationships across talkers for both consonants and vowels. Listeners identified words with varying potential acoustic-phonemic ambiguity across talkers (e.g., beet/boat vs. boot/boat) spoken by single or mixed talkers. Auditory categorization of words was always slower when listening to mixed talkers compared to a single talker, even when there was no potential acoustic ambiguity between target sounds. Moreover, the processing cost imposed by mixed talkers was greatest when words had the most potential acoustic-phonemic overlap across talkers. Models of acoustic dissimilarity between target speech sounds did not account for the pattern of results. These results suggest (a) that talker normalization incurs the greatest processing cost when disambiguating highly confusable sounds and (b) that talker normalization appears to be an obligatory component of speech perception, taking place even when the acoustic-phonemic relationships across sounds are unambiguous.

EEG oscillations during word processing predict MCI conversion to Alzheimer's disease.

Only a subset of mild cognitive impairment (MCI) patients progress to develop a form of dementia. A prominent feature of Alzheimer's disease (AD) is a progressive decline in language. We investigated if subtle anomalies in EEG activity of MCI patients during a word comprehension task could provide insight into the likelihood of conversion to AD. We studied 25 amnestic MCI patients, a subset of whom developed AD within 3-years, and 11 elderly controls. In the task, auditory category descriptions (e.g., ‘a type of wood’) were followed by a single visual target word either semantically congruent (i.e., oak) or incongruent with the preceding category. We found that the MCI convertors group (i.e. patients that would go on to convert to AD in 3-years) had a diminished early posterior-parietal theta (3–5 Hz) activity induced by first presentation of the target word (i.e., access to lexico-syntactic properties of the word), compared to MCI non-convertors and controls. Moreover, MCI convertors exhibited oscillatory signatures for processing the semantically congruent words that were different from non-convertors and controls. MCI convertors thus showed basic anomalies for lexical and meaning processing. In addition, both MCI groups showed anomalous oscillatory signatures for the verbal learning/memory of repeated words: later alpha suppression (9–11 Hz), which followed first presentation of the target word, was attenuated for the second and third repetition in controls, but not in either MCI group. Our findings suggest that a subtle breakdown in the brain network subserving language comprehension can be foretelling of conversion to AD.

Functional Preference for Object Sounds and Voices in the Brain of Early Blind and Sighted Individuals

Sounds activate occipital regions in early blind individuals. However, how different sound categories map onto specific regions of the occipital cortex remains a matter of debate. We used fMRI to characterize brain responses of early blind and sighted individuals to familiar object sounds, human voices, and their respective low-level control sounds. In addition, sighted participants were tested while viewing pictures of faces, objects, and phase-scrambled control pictures. In both early blind and sighted, a double dissociation was evidenced in bilateral auditory cortices between responses to voices and object sounds: Voices elicited categorical responses in bilateral superior temporal sulci, whereas object sounds elicited categorical responses along the lateral fissure bilaterally, including the primary auditory cortex and planum temporale. Outside the auditory regions, object sounds also elicited categorical responses in the left lateral and in the ventral occipitotemporal regions in both groups. These regions also showed response preference for images of objects in the sighted group, thus suggesting a functional specialization that is independent of sensory input and visual experience. Between-group comparisons revealed that, only in the blind group, categorical responses to object sounds extended more posteriorly into the occipital cortex. Functional connectivity analyses evidenced a selective increase in the functional coupling between these reorganized regions and regions of the ventral occipitotemporal cortex in the blind group. In contrast, vocal sounds did not elicit preferential responses in the occipital cortex in either group. Nevertheless, enhanced voice-selective connectivity between the left temporal voice area and the right fusiform gyrus were found in the blind group. Altogether, these findings suggest that, in the absence of developmental vision, separate auditory categories are not equipotent in driving selective auditory recruitment of occipitotemporal regions and highlight the presence of domain-selective constraints on the expression of cross-modal plasticity.

Perceptual Plasticity for Auditory Object Recognition.

In our auditory environment, we rarely experience the exact acoustic waveform twice. This is especially true for communicative signals that have meaning for listeners. In speech and music, the acoustic signal changes as a function of the talker (or instrument), speaking (or playing) rate, and room acoustics, to name a few factors. Yet, despite this acoustic variability, we are able to recognize a sentence or melody as the same across various kinds of acoustic inputs and determine meaning based on listening goals, expectations, context, and experience. The recognition process relates acoustic signals to prior experience despite variability in signal-relevant and signal-irrelevant acoustic properties, some of which could be considered as “noise” in service of a recognition goal. However, some acoustic variability, if systematic, is lawful and can be exploited by listeners to aid in recognition. Perceivable changes in systematic variability can herald a need for listeners to reorganize perception and reorient their attention to more immediately signal-relevant cues. This view is not incorporated currently in many extant theories of auditory perception, which traditionally reduce psychological or neural representations of perceptual objects and the processes that act on them to static entities. While this reduction is likely done for the sake of empirical tractability, such a reduction may seriously distort the perceptual process to be modeled. We argue that perceptual representations, as well as the processes underlying perception, are dynamically determined by an interaction between the uncertainty of the auditory signal and constraints of context. This suggests that the process of auditory recognition is highly context-dependent in that the identity of a given auditory object may be intrinsically tied to its preceding context. To argue for the flexible neural and psychological updating of sound-to-meaning mappings across speech and music, we draw upon examples of perceptual categories that are thought to be highly stable. This framework suggests that the process of auditory recognition cannot be divorced from the short-term context in which an auditory object is presented. Implications for auditory category acquisition and extant models of auditory perception, both cognitive and neural, are discussed.

When music is salty: The crossmodal associations between sound and taste.

Here we investigate associations between complex auditory and complex taste stimuli. A novel piece of music was composed and recorded in four different styles of musical articulation to reflect the four basic tastes groups (sweet, sour, salty, bitter). In Experiment 1, participants performed above chance at pairing the music clips with corresponding taste words. Experiment 2 uses multidimensional scaling to interpret how participants categorize these musical stimuli, and to show that auditory categories can be organized in a similar manner as taste categories. Experiment 3 introduces four different flavors of custom-made chocolate ganache and shows that participants can match music clips with the corresponding taste stimuli with above-chance accuracy. Experiment 4 demonstrates the partial role of pleasantness in crossmodal mappings between sound and taste. The present findings confirm that individuals are able to make crossmodal associations between complex auditory and gustatory stimuli, and that valence may mediate multisensory integration in the general population.

Attentional modulation and domain-specificity underlying the neural organization of auditory categorical perception.

Categorical perception (CP) is highly evident in audition when listeners' perception of speech sounds abruptly shifts identity despite equidistant changes in stimulus acoustics. While CP is an inherent property of speech perception, how (if) it is expressed in other auditory modalities (e.g., music) is less clear. Moreover, prior neuroimaging studies have been equivocal on whether attentional engagement is necessary for the brain to categorically organize sound. To address these questions, we recorded neuroelectric brain responses [event-related potentials (ERPs)] from listeners as they rapidly categorized sounds along a speech and music continuum (active task) or during passive listening. Behaviorally, listeners' achieved sharper psychometric functions and faster identification for speech than musical stimuli, which was perceived in a continuous mode. Behavioral results coincided with stronger ERP differentiation between prototypical and ambiguous tokens (i.e., categorical processing) for speech but not for music. Neural correlates of CP were only observed when listeners actively attended to the auditory signal. These findings were corroborated by brain-behavior associations; changes in neural activity predicted more successful CP (psychometric slopes) for active but not passively evoked ERPs. Our results demonstrate auditory categorization is influenced by attention (active>passive) and is stronger for more familiar/overlearned stimulus domains (speech>music). In contrast to previous studies examining highly trained listeners (i.e., musicians), we infer that (i) CP skills are largely domain-specific and do not generalize to stimuli for which a listener has no immediate experience and (ii) categorical neural processing requires active engagement with the auditory stimulus.

Classifying song and speech: effects of focal temporal lesions and musical disorder

ABSTRACTSong and speech represent two auditory categories the brain usually classifies fairly easily. Functionally, this classification ability may depend to a great extent on characteristic features of pitch patterns present in song melody and speech prosody. Anatomically, the temporal lobe (TL) has been discussed as playing a prominent role in the processing of both. Here we tested individuals with congenital amusia and patients with unilateral left and right TL lesions in their ability to categorize song and speech. In a forced-choice paradigm, specifically designed auditory stimuli representing sung, spoken and “ambiguous” stimuli (being perceived as “halfway between” song and speech), had to be classified as either “song” or “speech”. Congenital amusics and TL patients, contrary to controls, exhibited a surprising bias to classifying the ambiguous stimuli as “song” despite their apparent deficit to correctly process features typical for song. This response bias possibly reflects a strategy where, based on available context information (here: forced choice for either speech or song), classification of non-processable items may be achieved through elimination of processable classes. This speech-based strategy masks the pitch processing deficit in congenital amusics and TL lesion patients.