Auditory Recognition Research Articles

On the clinical entity in audiology: (central) auditory processing and speech recognition in noise disorders.

The area of (central) auditory processing disorder [(C)APD] is highly controversial. This controversy is fueled in part by multiple ambiguous descriptions of this condition. The American Speech-Language and Hearing Association and the American Academy of Audiology state that (C)APD is a clinical entity, but other sources say that this is questionable. The clinical entity is a concept used to define the nature of a disorder in order to facilitate diagnosis and intervention. Nosography is the study of the systematic description of diseases and disorders. The work in the field of nosography by physicians Thomas Sydenham and Otto Guttentag is reviewed. The Sydenham-Guttentag criteria for the clinical entity are presented as a framework for the conceptualization of disorders in the field of audiology. This paper discusses central auditory processing and speech recognition-in-noise disorders in reference to these criteria.

Auditory tasks for assessment of sensory function and affective prosody in schizophrenia

Schizophrenia patients exhibit impairments in auditory-based social cognition, indicated by deficits in detection of prosody, such as affective prosody and basic pitch perception. However, little is known about the psychometric properties of behavioral tests used to assess these functions. The goal of this paper is to characterize the properties of prosody and pitch perception tasks and to investigate whether they can be shortened. The pitch perception test evaluated is a tone-matching task developed by Javitt and colleagues (J-TMT). The prosody test evaluated is the auditory emotion recognition task developed by Juslin and Laukka (JL-AER). The sample includes 124 schizophrenia patients (SZ) and 131 healthy controls (HC). Properties, including facility and discrimination, of each item were assessed. Effects of item characteristics (e.g., emotion) were also evaluated. Shortened versions of the tests are proposed based on facility, discrimination, and/or ability of item characteristics to discriminate between patients and controls. Test–retest reliability is high for patients and controls for both the original and short forms of the J-TMT and JL-AER. Thus, the original as well as short forms of the J-TMT and JL-AER are suggested for inclusion in clinical trials of social cognitive and perceptual treatments. The development of short forms further increases the utility of these auditory tasks in clinical trials and clinical practice. The large SZ vs. HC differences reported here also highlight the profound nature of auditory deficits and a need for remediation.

Deficits in auditory frequency discrimination and speech recognition in cochlear implant users

ObjectivesSpeech recognition varies considerably following cochlear implantation for reasons that are still poorly understood. Considering the role of frequency discrimination in normal speech recognition, the aim of this study was to investigate the association between deficits in auditory frequency discrimination and speech recognition in cochlear implant users.MethodsFrequency discrimination thresholds and speech recognition were assessed in a group of 20 cochlear implant users and 16 normally hearing controls.ResultsBased on their results on the speech recognition task, the cochlear implant users were categorized either as proficient (n = 10) or non-proficient users (n = 10). The non-proficient cochlear implant users had poorer auditory frequency discrimination compared to the normal hearing participants and proficient cochlear implant users (both P < 0.05). No significant difference was found between the proficient cochlear implant users and the normally hearing group (P > 0.05). Furthermore, a bivariate correlation analysis revealed a relationship between speech recognition and frequency discrimination.ConclusionsThe present findings suggest an association between auditory frequency discrimination and speech recognition proficiency in cochlear implant users. Although no causal link can be drawn from these data, possible reasons for this association are discussed.

A hierarchical model of vision (HMAX) can also recognize speech

HMAX is a well-known computational model of visual recognition in cortex consisting of just two computational operations – a “template match” and non-linear pooling – alternating in a feedforward hierarchy in which receptive fields exhibit increasing specificity and invariance [1]. Interestingly, auditory recognition problems (such as speech recognition) share similar computational requirements, and recent work in auditory neuroscience suggests that auditory and visual cortex share similar anatomical and functional organization. Based on these similarities, we tested whether HMAX could support an auditory recognition task (specifically, word spotting). To test HMAX on word spotting, recorded speech samples from the TIMIT corpus [2] were first converted into time-frequency spectrograms using a computational model of the auditory periphery [3]. These spectrograms were then split into 750 ms frames and input to a standard HMAX model [4]. Based on observed similarities between the receptive fields in primary auditory cortex (spectro-temporal receptive fields, or STRFs) and primary visual cortex (typically modeled as oriented Gabor filters), we used S1 filters identical to those used in vision [4]. Similarly, S2 “patches” were randomly selected from C1 representations of speech sounds drawn from an independent speech corpus. One vs. all linear support vector machines (SVMs) were then trained to discriminate frames that contain a target word from those that did not. These SVMs were then tested on a novel set of test sentences using a sliding frame approach (750 ms frame size, 20 ms step size). For each frame in a sentence, the SVM produced a distance from the hyperplane, and a threshold value was applied to produce a binary classification whether or not the target word was present in the sentence. When tested on target words that appeared in a fixed context (i.e. SA sentences in TIMIT), performance was highly robust, with ROC areas consistently above 0.9. When tested on target words that appeared in variable contexts (i.e., SI sentences in TIMIT), performance was somewhat decreased with ROC areas around 0.8. This decrease in performance is likely due to the inclusion of “clutter” (i.e., target irrelevant features) within the frame, also commonly observed when HMAX is applied to visual object recognition tasks [1]. These results are novel in that they provide support for the hypothesis that the simple computational framework implemented in HMAX – consisting of a feedforward hierarchy of only two alternating computational operations – may generalize beyond vision to support auditory recognition as well. It is possible that such a representation could give rise to stable neural encodings that are invariant to behaviorally irrelevant characteristics as seen in higher order visual and auditory cortices [5,6]. While it is likely that this auditory version of the HMAX model would benefit from the use of more auditory-specific filters based on STRF models [7], the Gabor features used here are largely compatible with previous computational models based on STRFs up to the level of primary auditory cortex [8]. Additional benefit may also be gained by learning sparse representations from natural sounds, at both the S1 and S2 levels [9].

The attentional boost effect with verbal materials.

Study stimuli presented at the same time as unrelated targets in a detection task are better remembered than stimuli presented with distractors. This attentional boost effect (ABE) has been found with pictorial (Swallow & Jiang, 2010) and more recently verbal materials (Spataro, Mulligan, & Rossi-Arnaud, 2013). The present experiments examine the generality of the ABE with verbal materials and critically assess the perceptual encoding hypothesis, the notion that the memory benefits are due to enhanced encoding of the perceptual properties of the study stimulus. Experiments 1 and 3 demonstrated an ABE with visual study items, comparable in size whether the recognition test was visual or auditory. Experiments 2 and 3 established an ABE for auditory study stimuli that was again equivalent for auditory and visual recognition tests. Experiments 4 and 5 found an ABE on the test of free recall. Finally, the ABE was greater for high-frequency than low-frequency words. The results demonstrate the generality of the ABE over study and test modality, and over memory tests (recognition and free recall), while also documenting a moderating factor (word frequency). Importantly, the representational basis for the ABE with verbal materials appears to be abstract, or amodal, rather than modality specific.

Recognition and production of emotions in children with cochlear implants

The aim of this study was to examine auditory recognition and vocal production of emotions in three prelingually bilaterally profoundly deaf children aged 6–7 who received cochlear implants before age 2, and compare them with age-matched normally hearing children. No consistent advantage was found for the normally hearing participants. In both groups, sadness was recognized best and disgust was the most difficult. Confusion matrices among other emotions (anger, happiness, and fear) showed that children with and without hearing impairment may rely on different cues. Both groups of children showed that perception is superior to production. Normally hearing children were more successful in the production of sadness, happiness, and fear, but not anger or disgust. The data set is too small to draw any definite conclusions, but it seems that a combination of early implantation and regular auditory–oral-based therapy enables children with cochlear implants to process and produce emotional content comparable with children with normal hearing.

Auditory acclimatization and hearing aids: late auditory evoked potentials and speech recognition following unilateral and bilateral amplification.

The aim of this study was to investigate changes in central auditory processing following unilateral and bilateral hearing aid fitting using a combination of physiological and behavioral measures: late auditory event-related potentials (ERPs) and speech recognition in noise, respectively. The hypothesis was that for fitted ears, the ERP amplitude would increase over time following hearing aid fitting in parallel with improvement in aided speech recognition. The N1 and P2 ERPs were recorded to 500 and 3000 Hz tones presented at 65, 75, and 85 dB sound pressure level to either the left or right ear. New unilateral and new bilateral hearing aid users were tested at the time of first fitting and after 12 weeks hearing aid use. A control group of long-term hearing aid users was tested over the same time frame. No significant changes in the ERP were observed for any group. There was a statistically significant 2% improvement in aided speech recognition over time for all groups, although this was consistent with a general test-retest effect. This study does not support the existence of an acclimatization effect observable in late ERPs following 12 weeks' hearing aid use.

Function determines structure in complex neural networks.

In the course of our daily lives, we rely on our ability to quickly recognize visual objects despite variations in the viewing angle or lighting conditions (1). Most of the time, this process occurs so naturally that it is easy to underestimate the computational challenge it poses. In fact, as much as one third of our cortex is involved in visual object recognition (2). Thus far, we have not been able to create artificial object recognition systems that can match the performance of the human visual system, especially when one takes into account the energetic constraints. However, understanding the principles of visual object recognition has major implications not only because of potential wide-ranging practical applications but also because these principles are likely to hold clues to how sensory systems work in general. After all, there is a growing body of experimental and computational evidence that similar principles (3) might be at work during visual, auditory, or olfactory object recognition tasks. In PNAS, Yamins et al. present an important advance in this research direction by demonstrating how to optimize artificial networks to match the discrimination ability of primates when it comes to categorizing objects (4). Importantly, optimized networks share many features with the corresponding networks in the brain. Thus, although the structure of neural networks in the brain could have been largely determined by the idiosyncrasies of the previous evolutionary trajectory, they seem to reflect instead a unique optimal solution. This in turn offers more support for theoretical and computational pursuits to find optimal circuit organization in the brain (5).

Amusia and protolanguage impairments in schizophrenia.

Both language and music are thought to have evolved from a musical protolanguage that communicated social information, including emotion. Individuals with perceptual music disorders (amusia) show deficits in auditory emotion recognition (AER). Although auditory perceptual deficits have been studied in schizophrenia, their relationship with musical/protolinguistic competence has not previously been assessed. Musical ability was assessed in 31 schizophrenia/schizo-affective patients and 44 healthy controls using the Montreal Battery for Evaluation of Amusia (MBEA). AER was assessed using a novel battery in which actors provided portrayals of five separate emotions. The Disorganization factor of the Positive and Negative Syndrome Scale (PANSS) was used as a proxy for language/thought disorder and the MATRICS Consensus Cognitive Battery (MCCB) was used to assess cognition. Highly significant deficits were seen between patients and controls across auditory tasks (p < 0.001). Moreover, significant differences were seen in AER between the amusia and intact music-perceiving groups, which remained significant after controlling for group status and education. Correlations with AER were specific to the melody domain, and correlations between protolanguage (melody domain) and language were independent of overall cognition. This is the first study to document a specific relationship between amusia, AER and thought disorder, suggesting a shared linguistic/protolinguistic impairment. Once amusia was considered, other cognitive factors were no longer significant predictors of AER, suggesting that musical ability in general and melodic discrimination ability in particular may be crucial targets for treatment development and cognitive remediation in schizophrenia.

Multisensory integration, learning, and the predictive coding hypothesis

The multimodal nature of perception has generated several questions of importance pertaining to the encoding, learning, and retrieval of linguistic representations (e.g., Summerfield, 1987; Altieri et al., 2011; van Wassenhove, 2013). Historically, many theoretical accounts of speech perception have been driven by descriptions of auditory encoding; this makes sense because normal-hearing listeners rely predominantly on the auditory signal. However, from both evolutionary and empirical standpoints, comprehensive neurobiological accounts of speech perception must account for interactions across sensory modalities and the interplay of cross-modal and articulatory representations. These include auditory, visual, and somatosensory modalities. In a recent review, van Wassenhove (2013) discussed key frameworks describing how visual cues interface with the auditory modality to improve auditory recognition (Sumby and Pollack, 1954), or otherwise contribute to an illusory percept for mismatched auditory-visual syllables (McGurk and MacDonald, 1976). These frameworks encompass multiple levels of analysis. Some of these higher cognitive processing models that discuss parallel processing (Altieri and Townsend, 2011) or the independent extraction of features from the auditory and visual modalities (Massaro, 1987, Fuzzy Logical Model of Perception), early feature encoding (van Wassenhove et al., 2005), and encoding/timing at the neural level (Poeppel et al., 2008; Schroeder et al., 2008). This commentary on van Wassenhove (2013) will examine predictive coding hypotheses as one theory for how visemes are matched with auditory cues. Crucially, a hypothesized role shall be emphasized for cross-modal neural plasticity and multisensory learning in reinforcing the sharing of cues across modalities into adulthood.

Auditory pattern recognition and steering in the cricket Teleogryllus oceanicus

AbstractMale crickets Teleogryllus oceanicus (Le Guillou) produce a complex species‐specific calling song with phrases combining groups of single pulses (chirps) and groups of double‐pulses (trills) to attract females, which fly or walk towards singing males. In open‐loop trackball experiments, phonotactic steering responses to normal calling song phrases consisting of chirps and trills are strongest, suggesting that both components are necessary for maximal attractiveness. Sequences of just chirps or trills are less effective in eliciting phonotactic walking and steering. Split‐song paradigms are used to analyze the steering behaviour underlying orientation in more detail. The females' phonotactic steering reflects the alternating acoustic pattern of the split‐song paradigm. Analysis with high temporal resolution demonstrate, that even when the calling song is presented only from one side, the steering velocity and lateral deviation towards the song is modulated by steering events to single‐sound pulses. Therefore, pattern recognition, which integrates the structure of the song, appears not to be directly involved in the rapid steering response. This organization of phonotactic behaviour with a parallel processing of pattern recognition and steering is similar to other cricket species and may allow T. oceanicus females to steer transiently towards distorted song patterns as they occur in natural habitats.

The role of morphology in phoneme prediction: Evidence from MEG

There is substantial neural evidence for the role of morphology (word-internal structure) in visual word recognition. We extend this work to auditory word recognition, drawing on recent evidence that phoneme prediction is central to this process. In a magnetoencephalography (MEG) study, we crossed morphological complexity (bruis-er vs. bourbon) with the predictability of the word ending (bourbon vs. burble). High prediction error (surprisal) led to increased auditory cortex activity. This effect was enhanced for morphologically complex words. Additionally, we calculated for each timepoint the surprisal corresponding to the phoneme perceived at that timepoint, as well as the cohort entropy, which quantifies the competition among words compatible with the string prefix up to that timepoint. Higher surprisal increased neural activity at the end of the word, and higher entropy decreased neural activity shortly after word onset. These results reinforce the role of morphology and phoneme prediction in spoken word recognition.

Unilateral Auditory Cortex Lesions Impair or Improve Discrimination Learning of Amplitude Modulated Sounds, Depending on Lesion Side

A fundamental principle of brain organization is bilateral symmetry of structures and functions. For spatial sensory and motor information processing, this organization is generally plausible subserving orientation and coordination of a bilaterally symmetric body. However, breaking of the symmetry principle is often seen for functions that depend on convergent information processing and lateralized output control, e.g. left hemispheric dominance for the linguistic speech system. Conversely, a subtle splitting of functions into hemispheres may occur if peripheral information from symmetric sense organs is partly redundant, e.g. auditory pattern recognition, and therefore allows central conceptualizations of complex stimuli from different feature viewpoints, as demonstrated e.g. for hemispheric analysis of frequency modulations in auditory cortex (AC) of mammals including humans. Here we demonstrate that discrimination learning of rapidly but not of slowly amplitude modulated tones is non-uniformly distributed across both hemispheres: While unilateral ablation of left AC in gerbils leads to impairment of normal discrimination learning of rapid amplitude modulations, right side ablations lead to improvement over normal learning. These results point to a rivalry interaction between both ACs in the intact brain where the right side competes with and weakens learning capability maximally attainable by the dominant left side alone.

Functional Connectivity between Face-Movement and Speech-Intelligibility Areas during Auditory-Only Speech Perception

It has been proposed that internal simulation of the talking face of visually-known speakers facilitates auditory speech recognition. One prediction of this view is that brain areas involved in auditory-only speech comprehension interact with visual face-movement sensitive areas, even under auditory-only listening conditions. Here, we test this hypothesis using connectivity analyses of functional magnetic resonance imaging (fMRI) data. Participants (17 normal participants, 17 developmental prosopagnosics) first learned six speakers via brief voice-face or voice-occupation training (<2 min/speaker). This was followed by an auditory-only speech recognition task and a control task (voice recognition) involving the learned speakers’ voices in the MRI scanner. As hypothesized, we found that, during speech recognition, familiarity with the speaker’s face increased the functional connectivity between the face-movement sensitive posterior superior temporal sulcus (STS) and an anterior STS region that supports auditory speech intelligibility. There was no difference between normal participants and prosopagnosics. This was expected because previous findings have shown that both groups use the face-movement sensitive STS to optimize auditory-only speech comprehension. Overall, the present findings indicate that learned visual information is integrated into the analysis of auditory-only speech and that this integration results from the interaction of task-relevant face-movement and auditory speech-sensitive areas.

Emotion recognition in preschool children: Associations with maternal depression and early parenting

Emotion knowledge in childhood has been shown to predict social functioning and psychological well-being, but relatively little is known about parental factors that influence its development in early childhood. There is some evidence that both parenting behavior and maternal depression are associated with emotion recognition, but previous research has only examined these factors independently. The current study assessed auditory and visual emotion recognition ability among a large sample of preschool children to examine typical emotion recognition skills in children of this age, as well as the independent and interactive effects of maternal and paternal depression and negative parenting (i.e., hostility and intrusiveness). Results indicated that children were most accurate at identifying happy emotional expressions. The lowest accuracy was observed for neutral expressions. A significant interaction was found between maternal depression and negative parenting behavior: children with a maternal history of depression were particularly sensitive to the negative effects of maladaptive parenting behavior on emotion recognition ability. No significant effects were found for paternal depression. These results highlight the importance of examining the effects of multiple interacting factors on children's emotional development and provide suggestions for identifying children for targeted preventive interventions.

Hearing words helps seeing words: A cross-modal word repetition effect

Watching a speaker say words benefits subsequent auditory recognition of the same words. In this study, we tested whether hearing words also facilitates subsequent phonological processing from visual speech, and if so, whether speaker repetition influences the magnitude of this word repetition priming. We used long-term cross-modal repetition priming as a means to investigate the underlying lexical representations involved in listening to and seeing speech. In Experiment 1, listeners identified auditory-only words during exposure and visual-only words at test. Words at test were repeated or new and produced by the exposure speaker or a novel speaker. Results showed a significant effect of cross-modal word repetition priming but this was unaffected by speaker changes. Experiment 2 added an explicit recognition task at test. Listeners’ lipreading performance was again improved by prior exposure to auditory words. Explicit recognition memory was poor, and neither word repetition nor speaker repetition improved it. This suggests that cross-modal repetition priming is neither mediated by explicit memory nor improved by speaker information. Our results suggest that phonological representations in the lexicon are shared across auditory and visual processing, and that speaker information is not transferred across modalities at the lexical level.

Shared perceptual processes in phoneme and word perception: Evidence from aphasia

Shared perceptual processes in phoneme and word perception: Evidence from aphasia

Auditory brainstem implants in neurofibromatosis Type 2: is open speech perception feasible?

Patients with bilateral auditory nerve destruction may perceive some auditory input with auditory brainstem implants (ABIs). Despite technological developments and trials in new stimulation sites, hearing is very variable and of limited quality. The goal of this study was to identify advantageous and critical factors that influence the quality of auditory function, especially speech perception. The authors conducted a prospective study on ABI operations performed with the aid of multimodality neuromonitoring between 2005 and 2009 in 18 patients with neurofibromatosis Type 2. Outcome was evaluated by testing word recognition (monotrochee-polysyllabic word test at auditory-only mode [MTPa]) and open speech perception (Hochmair-Schulz-Moser [HSM] sentence test), both in pure auditory mode. The primary outcome was the HSM score at 24 months. The predictive meaning of general clinical data, tumor volume, number of active electrodes, duration of deafness, and early hearing data was examined. In 16 successful ABI activations the average score for MTPa was 89% (SD 13%), and for HSM it was 41% (SD 32%) at 24 months. There were 2 nonresponders, 1 after radiosurgery and the other in an anatomical variant. Direct facial nerve reconstruction during the same surgery was followed by successful nerve recovery in 2 patients, with a simultaneous very good HSM result. Patients' age, tumor extension, and tumor volume were not negative predictors. There was an inverse relationship between HSM scores and deafness duration; 50% or higher HSM scores were found only in patients with ipsilateral deafness duration up to 24 months. The higher the deafness sum of both sides, the less likely that any HSM score will be achieved (p = 0.034). In patients with total deafness duration of less than 240 months, higher numbers of active electrodes were significantly associated with better outcomes. The strongest cross-correlation was identified between early MTPa score at 3 months and 24-month HSM outcome. This study documents that open-set speech recognition in pure auditory mode is feasible in patients with ABIs. Large tumor volumes do not prevent good outcome. Positive preconditions are short ipsilateral and short bilateral deafness periods and high number of auditory electrodes. Early ability in pure auditory word recognition tests indicates long-term capability of open speech perception.

Neural correlates of auditory recognition memory in the primate dorsal temporal pole

Temporal pole (TP) cortex is associated with higher-order sensory perception and/or recognition memory, as human patients with damage in this region show impaired performance during some tasks requiring recognition memory (Olson et al. 2007). The underlying mechanisms of TP processing are largely based on examination of the visual nervous system in humans and monkeys, while little is known about neuronal activity patterns in the auditory portion of this region, dorsal TP (dTP; Poremba et al. 2003). The present study examines single-unit activity of dTP in rhesus monkeys performing a delayed matching-to-sample task utilizing auditory stimuli, wherein two sounds are determined to be the same or different. Neurons of dTP encode several task-relevant events during the delayed matching-to-sample task, and encoding of auditory cues in this region is associated with accurate recognition performance. Population activity in dTP shows a match suppression mechanism to identical, repeated sound stimuli similar to that observed in the visual object identification pathway located ventral to dTP (Desimone 1996; Nakamura and Kubota 1996). However, in contrast to sustained visual delay-related activity in nearby analogous regions, auditory delay-related activity in dTP is transient and limited. Neurons in dTP respond selectively to different sound stimuli and often change their sound response preferences between experimental contexts. Current findings suggest a significant role for dTP in auditory recognition memory similar in many respects to the visual nervous system, while delay memory firing patterns are not prominent, which may relate to monkeys' shorter forgetting thresholds for auditory vs. visual objects.

Wernicke’s area revisited: Parallel streams and word processing

Auditory word-form recognition was originally proposed by Wernicke to occur within left superior temporal gyrus (STG), later further specified to be in posterior STG. To account for clinical observations (specifically paraphasia), Wernicke proposed his sensory speech center was also essential for correcting output from frontal speech-motor regions. Recent work, in contrast, has established a role for anterior STG, part of the auditory ventral stream, in the recognition of species-specific vocalizations in nonhuman primates and word-form recognition in humans. Recent work also suggests monitoring self-produced speech and motor control are associated with posterior STG, part of the auditory dorsal stream. Working without quantitative methods or evidence of sensory cortex’ hierarchical organization, Wernicke co-localized functions that today appear dissociable. “Wernicke’s area” thus may be better construed as two cortical modules, an auditory word-form area (AWFA) in the auditory ventral stream and an “inner speech area” in the auditory dorsal stream.