Auditory Search Research Articles

Investigating the role of visual and auditory search in reading and developmental dyslexia

It has been suggested that auditory and visual sequential processing deficits contribute to phonological disorders in developmental dyslexia. As an alternative explanation to a phonological deficit as the proximal cause for reading disorders, the visual attention span hypothesis (VA Span) suggests that difficulties in processing visual elements simultaneously lead to dyslexia, regardless of the presence of a phonological disorder. In this study, we assessed whether deficits in processing simultaneously displayed visual or auditory elements is linked to dyslexia associated with a VA Span impairment. Sixteen children with developmental dyslexia and 16 age-matched skilled readers were assessed on visual and auditory search tasks. Participants were asked to detect a target presented simultaneously with 3, 9, or 15 distracters. In the visual modality, target detection was slower in the dyslexic children than in the control group on a “serial” search condition only: the intercepts (but not the slopes) of the search functions were higher in the dyslexic group than in the control group. In the auditory modality, although no group difference was observed, search performance was influenced by the number of distracters in the control group only. Within the dyslexic group, not only poor visual search (high reaction times and intercepts) but also low auditory search performance (d′) strongly correlated with poor irregular word reading accuracy. Moreover, both visual and auditory search performance was associated with the VA Span abilities of dyslexic participants but not with their phonological skills. The present data suggests that some visual mechanisms engaged in “serial” search contribute to reading and orthographic knowledge via VA Span skills regardless of phonological skills. The present results further open the question of the role of auditory simultaneous processing in reading as well as its link with VA Span skills.

Searching for audiovisual correspondence in multiple speaker scenarios

A critical question in multisensory processing is how the constant information flow that arrives to our different senses is organized in coherent representations. Some authors claim that pre-attentive detection of inter-sensory correlations supports crossmodal binding, whereas other findings indicate that attention plays a crucial role. We used visual and auditory search tasks for speaking faces to address the role of selective spatial attention in audiovisual binding. Search efficiency amongst faces for the match with a voice declined with the number of faces being monitored concurrently, consistent with an attentive search mechanism. In contrast, search amongst auditory speech streams for the match with a face was independent of the number of streams being monitored concurrently, as long as localization was not required. We suggest that the fundamental differences in the way in which auditory and visual information is encoded play a limiting role in crossmodal binding. Based on these unisensory limitations, we provide a unified explanation for several previous apparently contradictory findings.

Revisiting Arieti's "Listening Attitude" and Hallucinated Voices

Silvano Arieti proposed that auditory/verbal hallucinations (AVHs) are triggered by momentary states of heightened auditory attention that he identified as a "listening attitude." Studies and clinical observations by our group support this view. Patients enrolled in our repetitive transcranial magnetic stimulation trials, if experiencing a significant curtailment of these hallucinations, often report an episodic sense that their voices are still occurring even if they no longer can be heard, suggesting episodic states of heightened auditory expectancy. Moreover, a functional magnetic resonance study reported by our group detected activation in the left insula prior to hallucination events. This finding is suggestive of activation in the same region detected in healthy subjects during "auditory search" in response to ambiguous sounds when anticipating meaningful speech. AVHs often are experienced with a deep emotional salience and may occur in the context of dramatic social isolation that together could reinforce heightened auditory expectancy. These findings and clinical observations suggest that Arieti's original formulation deserves further study.

Spatial gradient for unique-feature detection in patients with unilateral neglect: Evidence from auditory and visual search

Patients with unilateral spatial neglect following right hemisphere damage are impaired in detecting contralesional targets in both visual and haptic search tasks, and often show a graded improvement in detection performance for more ipsilesional spatial locations. In audition, multiple simultaneous sounds are most effectively perceived if they are distributed along the frequency dimension. Thus, attention to spectro-temporal features alone can allow detection of a target sound amongst multiple simultaneous distracter sounds, regardless of whether these sounds are spatially separated. Spatial bias in attention associated with neglect should not affect auditory search based on spectro-temporal features of a sound target. We report that a right brain damaged patient with neglect demonstrated a significant gradient favouring the ipsilesional side on a visual search task as well as an auditory search task in which the target was a frequency modulated tone amongst steady distractor tones. No such asymmetry was apparent in the auditory search performance of a control patient with a right hemisphere lesion but no neglect. The results suggest that the spatial bias in attention exhibited by neglect patients affects stimulus processing even when spatial information is irrelevant to the task.

The role of spatial location in auditory search

The majority of research findings to date indicate that spatial cues play a minor role in enhancing listeners’ ability to parse and detect a sound of interest when it is presented in a complex auditory scene comprising multiple simultaneous sounds. Frequency and temporal differences between sound streams provide more reliable cues for scene analysis as well as for directing attention to relevant auditory ‘objects’ in complex displays. The present study used naturalistic sounds with varying spectro-temporal profiles to examine whether spatial separation of sound sources can enhance target detection in an auditory search paradigm. The arrays of sounds were presented in virtual auditory space over headphones. The results of Experiment 1 suggest that target detection is enhanced when sound sources are spatially separated relative to when they are presented at the same location. Experiment 2 demonstrated that this effect is most prominent within the first 250 ms of exposure to the array of sounds. These findings suggest that spatial cues may be effective for enhancing early processes such as stream segregation, rather than simply directing attention to objects that have already been segmented.

Overriding auditory attentional capture

Attentional capture by color singletons during shape search can be eliminated when the target is not a feature singleton (Bacon & Egeth, 1994). This suggests that a "singleton detection" search strategy must be adopted for attentional capture to occur. Here we find similar effects on auditory attentional capture. Irrelevant high-intensity singletons interfered with an auditory search task when the target itself was also a feature singleton. However, singleton interference was eliminated when the target was not a singleton (i.e., when nontargets were made heterogeneous, or when more than one target sound was presented). These results suggest that auditory attentional capture depends on the observer's attentional set, as does visual attentional capture. The suggestion that hearing might act as an early warning system that would always be tuned to unexpected unique stimuli must therefore be modified to accommodate these strategy-dependent capture effects.

Performance of auditory search with Japanese speech by native speakers and its learners as L2

Auditory search experiments using normal Japanese speech (NJS) and time-reversed speech (TRS) [Asemi et al., Acoust. Sci. Technol., 24, 145–147 (2003)] demonstrated fast and accurate judgment of the presence or absence of the target NJS among distracting TRSs. For application of an auditory search to measurement of the ability of auditory comprehension of foreign language, the effects of the learning proficiency of Japanese on the speed and accuracy in the detection of the target NJS were examined through auditory search experiments for native and non-native speakers of Japanese. The experiment comprised two parts: an NJS was the target and TRSs were distractors and vice versa. Subjects were requested to judge the presence or absence of a target speech correctly and as quickly as possible when a target and several distracters were presented simultaneously and diotically. Results showed remarkable differences between the performance of natives and non-native speakers in error rates rather than in the reaction times. Moreover, significant correlations were found between the error rates and the learning experiences only when fewer than three utterances were presented simultaneously. [Work supported by MEXT.]

Roles of frequency information and spatial information in auditory search

Effects of attention on auditory search process were examined with five spatially distributed loudspeakers, each of which radiated a component sound (either a pure tone or a 1/12-octave NBN). The stimulus consisted of five component sounds with different (center) frequencies. The listener’s task was to judge whether or not a target sound was included in the distractors as quickly as possible while minimizing errors [N. Asemi et al., Acta Acustica 89, 346–354 (2003)]. The following three conditions were examined: (1) a control condition with no cue sound; (2) a condition with a preceding cue sound indicating the target sound location; and (3) a probe-signal condition in which 1/8 of the cue sounds provided inconsistent information of frequency and location (probes) while the remainder provided consistent information (primaries). The resultant response times for probes were generally longer than those for primaries. Furthermore, attention windows were observed in both frequency and spatial domains. The effect of the disparities between the cue and target was larger when both the frequency and location differed than when only one of them differed. This result also means that the detection performance becomes better if attention is directed with certain information related to the targets.

Quantifying Frequency Dependence of Auditory Search

In the present study, we examined the frequency dependence of auditory search performance. Detection thresholds were measured for an 800-Hz target tone in a sequence of distractor tones (informational masking) as a function of frequency separation between the target and the distractor tones. The results showed that the thresholds decreased monotonically with frequency separation increasing. To further quantify the frequency dependence of auditory search performance, we applied a roex function (Patterson and Moore, 1986) to estimate a filter bandwidth for the threshold data. The estimated bandwidth was wider than that of an auditory filter by a factor of five (Leek et al., 1991). This result, together with some earlier results (Mori, 2003), demonstrates the effectiveness of the informational masking we used.

Auditory search asymmetry between normal Japanese speech sounds and time-reversed speech sounds distributed on the frontal-horizontal plane

Auditory search asymmetry between normal Japanese speech sounds and time-reversed speech sounds distributed on the frontal-horizontal plane

The Effects of Visual Cues and Interstimulus Interval on Accuracy in Auditory Localization and Detection

Auditory localization is an increasingly important topic as the technology for audio displays is becoming more available. However, few studies examine the effects of multiple simultaneous distracters on auditory detection and localization performance. Previous research has found that detection and localization performance significantly drop as the number of distracters increases; however, it is not clear what causes these errors. In the present study, participants either had to localize an auditory stimulus or detect an auditory stimulus among multiple distracters. Similar to previous research the number of errors significantly increased as the number of active speakers increased. Also consistent with previous research, the detection performance was better than the localization performance. The use of visual cues was found to benefit the localization group but did not significantly affect performance in the detection group. The present study also found that a longer interstimulus interval improved accuracy only in the localization group, and then only when visual cues were present. These findings provide insight into the complex nature of the auditory search task.

The Effects of Speaker Gender, Frequency, and Distinctiveness on Auditory Search Performance

The purpose of this study was to investigate the influence of speaker gender, frequency content and distinctiveness on performance in an auditory search task. Male and female voices were manipulated and combined to create several conditions that examined distinctiveness, distracter heterogeneity, and artificially altered sound frequency. Participants were required to monitor four simultaneous streams of auditory data consisting of digits and letters. Participants listened for a target and then indicated which speaker presented the target. No significant differences were found between male and female targets. Participant's ability to localize targets was greatest when targets were distinct from their distracters. This distinctiveness occurred when either gender or frequency differences between the target and distracters were present.

Auditory search asymmetry with pure tone, narrow-band noise, amplitude-modulated tone, and frequency-modulated tone

To examine basic features of sounds used in the perception of sound environment, auditory search tasks seem promising to us. Thus the response time to detect a target sound among distracting sound(s) was measured by three experiments. In the first experiment, pure tones and narrow-band noises (1/12 oct) were used. Either of them was used as a target and the other was used as distractor(s). The number of the distractor was varied from 1 to 6. Each sound was presented from one of seven loudspeakers located on the horizontal plane. In the second and third experiments, pure tones and amplitude-modulated tones, pure tones and frequency-modulated tones, respectively, were used as either a target or distractor(s). Results show that the response time to detect a narrow-band noise, an amplitude-modulated tone, or a frequency-modulated tone among distracting pure tones was hardly affected by the number of the distractors. However, the time required to detect a pure tone increased with the number of distractors. These results indicated that our auditory system utilizes temporal changes in amplitude and frequency of sound as a basic feature for the detection of a sound in a sound environment. Precise experimental results as well as further consideration will be given in the presentation.

Multi-Channel Auditory Search: Toward Understanding Control Processes in Polychotic Auditory Listening

Two experiments are presented which develop a framework for exploring auditory information processing. The framework is referred to as polychotic listening or auditory search, and it requires a listener to aurally scan multiple simultaneous auditory streams for the appearance of a target. The first experiment examined the participant's ability to aurally scan between two and six simultaneous auditory streams of letter and digit names for the name of a target letter. The main independent variable was auditory load, or the number of active audio streams. The primary dependent variable was accuracy. Accuracy and sensitivity measures showed that as load increased, performance decreased. The second study required participants to practice the same task for ten sessions (1,800 trials). Results indicated that even with extensive practice, performance was still affected by auditory load. The relation between the present results and similar results in the visual search literature are discussed.

PET Evidence for an Amodal Verbal Working Memory System

Current models of verbal working memory assume that modality-specific representations are translated into phonological representations before entering the working memory system. We report an experiment that tests this assumption. Positron emission tomography measures were taken while subjects performed a verbal working memory task. Stimuli were presented either visually or aurally, and a visual or auditory search task, respectively, was used as a control. Results revealed an almost complete overlap between the active memory areas regardless of input modality. These areas included dorsolateral frontal, Broca's area, SMA, and premotor cortex in the left hemisphere; bilateral superior and posterior parietal cortices and anterior cingulateright cerebellum. These results correspond well with previous research and suggest that verbal working memory is modality independent and is mediated by a circuit involving frontal, parietal, and cerebellar mechanisms.

Construct validity of neuropsychological tests of conceptual and attentional abilities

In a mixed sample of community-living adults, this study examined the construct validity of five neuropsychological tests: Category Test (CAT), Wisconsin Card Sorting Test (WCST), Paced Auditory Serial Addition Task (PASAT), Visual Search and Attention Test (VSAT) and Trail Making Test: Part B (TMT-B). Principal components analyses showed that PASAT, VSAT, and TMT-B defined an attention factor and that CAT and WCST defined a conceptual factor. The results were discussed in terms of common interpretations of these procedures as well as in terms of Mirsky's (1989) components of attention.

Selective Auditory - Visual Interference Effects in a Dual Task Paradigm: Implications for Assessing Individual Differences

Three experiments are reported that tested predictions from a multiple resource framework for selective patterns of interference in a visual - auditory dual task paradigm. Subjects performed an auditory memory search task both alone and in conjunction with several visual information processing tasks. In Experiment 1 the level of speech intelligibility in the memory search task did not affect performance in an unstable tracking task. In Experiments 2 and 3 speech intelligibility did affect performance in a spatial processing task and a mathematical reasoning task. These results are discussed in terms of their relevance for designing a test battery to identify individuals likely to be affected by changes in speech intelligibility.

Auditory Search Using Vowel Sounds

Using an auditory memory search task and 8 vowels as stimuli, latencies and accuracy (number correct) for 6 subjects were measured. Accuracy during the first session was above chance but evidence of automatic processing was tenuous and was not achieved for RT. Auditory search seems more difficult than visual search.

Auditory search for syllables embedded within meaningful sentences.

Sentence length, syllable position within a sentence, and grammatical class of word in which syllables were embedded were investigated with respect to their effects upon accuracy of syllable identification and response latency in a task of auditory search. Ninety-six university students listened to sentences, which were followed by a syllable that identified the target for search, and determined whether or not the identifying syllable appeared within the sentence preceding it. The time required for search was recorded in milli-seconds. Results indicate a strong negative relationship between accuracy of syllable identification and time required for search. Greatest accuracy and shortest response latencies occurred when syllables were embedded within the shortest sentences of the series (15 to 25 syllables in length), within the final fourth of a sentence, and within nouns, verbs, conjunctions, or prepositions rather than within adjectives and adverbs. Indications are that the task of auditory search requires a rapid, low level, minimal form of language processing and that semantic and syntactic clues are of major importance to successful, rapid processing. Theoretical concepts of selection, recoding, retention, and recall of auditory imputs are discussed in relation to the normal, daily processing of language.

Studies in Auditory Search

A previous paper presented at an ASA meeting (June, 1966) treated auditory search, involving the identification of syllables embedded within sentences. Results indicated relationships among lengths, syllable positions, word class in which syllables were embedded, and search time and accuracy of search. This paper extends the previous treatment and examines specifically several aspects of auditory search. Search time for the conditions in which syllables are present and absent were examined. Correct responses were always associated with shorter response latencies when the syllable was present; when the syllable was absent, search time for correct and incorrect responses were similar, with no consistent trend emerging. Examination of word class within which target syllables were located indicated that accuracy was poorer and search time longer for modifiers than for nouns, verbs, and connecting words. Although accuracy was similar for the latter three, time of search varied, indicating that search time may be a precise measurement for studies of this nature, and that the auditory processing of language may be greatly dependent on grammatical word class. Theoretical concepts relative to the possibility of different levels of language processing and the rôle of immediate memory in recall in the task of auditory search is discussed.