Environmental Sound Recognition Research Articles

Methods and Preliminary Outcomes of Pediatric Auditory Brainstem Implantation

The objective was to provide information about methods used and preliminary outcomes for pediatric ABI (auditory brainstem implant). An analysis of outcome was performed in children who received an ABI. Twelve children received a MED-EL ABI system. Progress in audition and language was monitored through parental reports, questionnaires, profiles, and closed-set tests. The median number of active electrodes was 9 of 12. Seven of 12 users consistently respond to sound, and 5 of 12 do not. Highest performers can recognize words in small sets and have begun to use some words. Auditory brainstem implants appear to be beneficial for some pediatric patients who cannot benefit from traditional cochlear implant surgery. Benefits in the short-term can be recognition of environmental sounds, recognition of some words and very commonly used phrases, and the beginning use of words. Although some of our ABI users demonstrate no response to sound, they do want to wear their sound processors all waking hours. The cause of lack of response may be related to the second intervention, which might have led to displacement of the electrode array, or presence of additional handicaps or syndromes. However, the results are less than optimal. The relatively short postoperative follow-up duration is a considered weakness of this study.

Environmental sound recognition by timbre in children with Williams syndrome

Anecdotal reports have described children with Williams syndrome (WS) as presenting outstanding skills for recognizing environmental sounds by their timbre. This has led to suggest that the skills for environmental sound recognition by timbre are highly developed in WS. Furthermore, the term hypertimbria has been proposed to refer to this feature. However, no academic research has assessed these skills in WS. This study therefore aimed to contrast the reports on the highly developed skills for environmental sound recognition by timbre in children with WS. An environmental sound recognition task was administered to children with WS, children with Down syndrome of the same chronological age and cognitive level, and chronological age-matched typically developing children. Participants with WS performed significantly lower than their typically developing peers and no significant differences were found between the WS and Down syndrome groups. Unlike previous reports, this study points out that in WS environmental sound recognition by timbre does not constitute a phenotypic strength either in absolute or relative terms. Results suggest that children with WS do not present hypertimbria or preserved skills for timbre recognition. We discuss the implications of these results for theories of cognitive modularity.

1A1-O04 環境音認識を用いたモジュラーホームロボットによる朝食準備作業(ホーム&オフィスロボット)

The aim of our research is development of a modular home robot. Task of the robot is managed by timetable. Timetable needs to estimate time that is longer than each task, because there is range in end time of task. So, we focus attention on environmental sounds associated with status and changes in environment. This paper describes implementation of breakfast table setting by a modular home robot with environmental sounds recognition.

The management of cochlear nerve deficiency

The assessment process is critical in deciding whether a profoundly deaf child with cochlear nerve deficiency (CND) will be suitable for a cochlear or auditory brainstem implant (ABI). Magnetic resonance imaging (MRI) using submillimetric T2 weighted gradient echo or turbo spin echo sequences is mandatory for all profoundly deaf children to diagnose CND. Evidence of audition on behavioural or electrophysiological tests following both auditory and electrical stimulation sometimes allows identification of significant auditory tissue not visible on MRI. In particular electric auditory brainstem response (EABR) testing may allow some quantification of auditory tissue and help decide whether a cochlear implant will be beneficial. Age and cognitive development are the most critical factors in determining ABI benefit.Hearing outcomes from both cochlear implants and ABIs are variable and likely to be limited in children with CND. A proportion of children will get no benefit. Usually the implants would be expected to provide recognition of environmental sounds and understanding of simple phonetics. Most children will not develop normal speech and they will often need to learn to communicate with sign language.The ABI involves a major neurosurgical procedure and at present the long term outcomes are unknown. It is therefore essential that parents who are considering this intervention have plenty of time to consider all aspects and the opportunity for in depth discussion.

Auditory Processing Following Consecutive Right Temporal Lobe Resections: A Prospective Case Study

The role of the right temporal lobe in processing speech is not well understood. Although the left temporal lobe has long been recognized as critical for speech perception, there is growing evidence for right hemisphere involvement. To investigate whether the right temporal lobe is critical for auditory speech processing, we studied prospectively a normal-hearing patient who underwent consecutive right temporal lobe resections for treatment of medically intractable seizures. To test the hypothesis that the right temporal lobe is critical for auditory speech processing. We used a prospective, repeated-measure, single-case design. Auditory processing was evaluated using behavioral tests of speech recognition (words, sentences) under multiple listening conditions (e.g., quiet, background noise, etc.). Auditory processing of nonspeech sounds was measured by pitch pattern sequencing and environmental sound recognition tasks. Repeat behavioral testing was performed at four time points over a 2 yr period: before and after consecutive right temporal lobe resection surgeries. Before surgery, the patient demonstrated normal speech recognition in quiet and under real-world listening conditions (background noise, filtered speech). After the initial right anterior temporal resection, speech recognition scores declined under adverse listening conditions, especially for the left ear, but remained largely within normal limits. Following resection of the right superior temporal gyrus 1 yr later, speech recognition in quiet and nonspeech sound processing (pitch patterns, environmental sounds) remained intact. However, speech recognition under adverse listening conditions was severely impaired. The right superior temporal gyrus appears to be critical for auditory processing of speech under real-world listening conditions.

Environmental Sounds Recognition in Children with Cochlear Implants

The aims of this study were (1) to document the recognition performance of environmental sounds (ESs) in Mandarin-speaking children with cochlear implants (CIs) and to analyze the possible associated factors with the ESs recognition; (2) to examine the relationship between perception of ESs and receptive vocabulary level; and (3) to explore the acoustic factors relevant to perceptual outcomes of daily ESs in pediatric CI users. Forty-seven prelingually deafened children between ages 4 to 10 years participated in this study. They were divided into pre-school (group A: age 4–6) and school-age (group B: age 7 to 10) groups. Sound Effects Recognition Test (SERT) and the Chinese version of the revised Peabody Picture Vocabulary Test (PPVT-R) were used to assess the auditory perception ability. The average correct percentage of SERT was 61.2% in the preschool group and 72.3% in the older group. There was no significant difference between the two groups. The ESs recognition performance of children with CIs was poorer than that of their hearing peers (90% in average). No correlation existed between ESs recognition and receptive vocabulary comprehension. Two predictive factors: pre-implantation residual hearing and duration of CI usage were found to be associated with recognition performance of daily-encountered ESs. Acoustically, sounds with distinct temporal patterning were easier to identify for children with CIs. In conclusion, we have demonstrated that ESs recognition is not easy for children with CIs and a low correlation existed between linguistic sounds and ESs recognition in these subjects. Recognition ability of ESs in children with CIs can only be achieved by natural exposure to daily-encountered auditory stimuli if sounds other than speech stimuli were less emphasized in routine verbal/oral habilitation program. Therefore, task-specific measures other than speech materials can be helpful to capture the full profile of auditory perceptual progress after implantation.

Very short time environmental sound classification based on spectrogram pattern matching

Environmental sounds are unstructured and similar to noise. However, the recognition of environmental sounds can benefit crime investigations, warning systems for elderly persons, and security systems. A few past research projects were developed for classifying the environmental sounds. In this paper, we proposed an environmental sound classification algorithm using spectrogram pattern matching along with neural network and k-nearest neighbor (k-NN) classifiers. Unlike other techniques, our approach is based on the observation that local features are more important than global features. In addition, our technique can avoid the problem of filtering less informative and irrelevant frequencies in the classification step. Twenty types of sound from BBC and Sound Ideas databases, with each sound sample longer than 10min, were tested with our algorithm. The spectrogram feature was compared with mel frequency delta cepstral coefficient (MFCC), linear prediction coefficient (LPC), and matching pursuit (MP) features. Two relevant factors concerning the accuracy of classification, window size and sampling rate, were also investigated to find the suitable value of each factor. We also investigated all combinations of these features. Using the k-NN classifier, the maximum accuracy of 94.98% occurred when the spectrogram, LPC, and MP features were combined.The experiments showed that the spectrogram feature with a feed-forward neural network can effectively classify 1-s audio clips with an accuracy of 85.66%. Furthermore, a longer duration can significantly increase the accuracy of classifying each sound clip to a certain limit without affecting cost. Using an audio clip duration of 6s, the spectrogram feature with a feed-forward neural network provided the best classification accuracy, at least 90.57%.

Recognition of environmental sound recorded by mobile phone

Recognizing and understanding of environmental sounds shall give higher semantics of the surrounding context and situation. This paper presents a proposal of an innovative environmental sound recognition method as a combination of robust ICA basis in a frequency domain and robust MP feature in the time domain. Implementation details and its performance evaluations are presented. Its feasibility and future works are provided.

Auditory brainstem implants for neurofibromatosis type 2

Neurofibromatosis type 2 (NF2) produces benign Schwann cell tumors on many cranial nerves, in particular on the vestibular portions of the VIIIn bilaterally. Removal of these vestibular schwannomas usually severs the auditory portion of the VIIIn, thus deafening the patients. The auditory brainstem implant (ABI) was designed to provide prosthetic electric stimulation of the cochlear nucleus in the brainstem to restore some hearing sensations to patients deafened by bilateral removal of vestibular schwannomas. This study will review the new developments and improving outcomes of the ABI. From its initial application in 1979 until about 2005, the ABI provided modest but useful auditory sensations to NF2 patients. However, application of the ABI in non-NF2 populations and in children with congenital malformations demonstrated better results, showing that the ABI could provide high levels of speech recognition. Recent results show excellent speech recognition in NF2 patients as well. This study will discuss the potential causes of the variability in ABI outcomes. ABIs activate neurons in the cochlear nucleus to recreate hearing sensations in people who have become deaf as a result of the loss of the auditory nerve. Most NF2 patients show functional hearing benefit from the ABI, with awareness and recognition of environmental sounds and enhancement of lipreading. It is now clear that ABIs can produce excellent speech recognition in some patients with NF2, allowing even conversational telephone use. Although the factors leading to this improved performance are not completely clear, these new results show that excellent hearing is possible for NF2 patients with the ABI.

A study of the Speech and Environmental Sound Recognition in the Classroom Noise for School-aged Children with Hearing Loss

일상생활 속에서 현대인은 자동차의 경적음, 컴퓨터의 팬 돌아가는 소리, 공사장 소음 등 여러 가지 크고 작은 소 음을 경험하게 되는데, 실생활 속 소음은 방음실에서 녹음 된 소음에 비해 주파수, 시간적 정보 면에서 보다 복잡하고 강도와 스펙트럼 상 다변화하는 양상을 보인다(Crandell & Bess, 1986; Rhebergen et al., 2008). 아동의 경우 성인 보다 소음 속에서 말소리를 듣고 이해하는 것이 어렵고 (Elliott et al., 1979; Fallon et al., 2000; Nittrouer & Boothroyd, 1990; Picard & Bradley, 2001), 특히 건청 아동보다 난청아동이 소음에 의한 영향을 더욱 많이 받게 되므로(Crandell & Smaldino, 2000) 실생활 속 난청아동 의 소음하 인지능력을 알기 위해서는 조용한 상황이 아닌 아동들이 주로 접하게 되는 소음환경에서 인지능력을 측정 하는 것이 중요하다.

NMF-based environmental sound source separation using time-variant gain features

Various environmental sounds exist around us in our daily life. Recently, environmental sound recognition has drawn great attention for understanding our environment. However, because environmental sounds derive from multiple sound sources, it is difficult to recognize them accurately. If we were able to separate sound sources before sound recognition as a pre-process, then recognition would be easier and more accurate. We assume that monaural microphones are widely installed in mobile devices used as recording devices. This paper therefore presents a proposal for monaural sound source separation of environmental sounds. Two-phase clustering using non-negative matrix factorization (NMF) is proposed to separate monaural sound sources. In this proposal, the time-variant gain feature is used as an attribute of an environmental sound for more efficient sound separation.

환경음 인식을 위한 GMM의 혼합모델 개수 추정

In this paper we applied the optimal mixture number estimation technique in GMM(Gaussian mixture model) using BIC(Bayesian information criterion) and MDL(minimum description length) as a model selection criterion for environmental sounds recognition. In the experiment, we extracted 12 MFCC(mel-frequency cepstral coefficients) features from 9 kinds of environmental sounds which amounts to 27747 data and classified them with GMM. As mentioned above, BIC and MDL is applied to estimate the optimal number of mixtures in each environmental sounds class. According to the experimental results, while the recognition performances are maintained, the computational complexity decreases by 17.8% with BIC and 31.7% with MDL. It shows that the computational complexity reduction by BIC and MDL is effective for environmental sounds recognition using GMM.

Environmental Sound Recognition Using Time-Frequency Intersection Patterns

Environmental sound recognition is an important function of robots and intelligent computer systems. In this research, we use a multistage perceptron neural network system for environmental sound recognition. The input data is a combination of time-variance pattern of instantaneous powers and frequency-variance pattern with instantaneous spectrum at the power peak, referred to as a time-frequency intersection pattern. Spectra of many environmental sounds change more slowly than those of speech or voice, so the intersectional time-frequency pattern will preserve the major features of environmental sounds but with drastically reduced data requirements. Two experiments were conducted using an original database and an open database created by the RWCP project. The recognition rate for 20 kinds of environmental sounds was 92%. The recognition rate of the new method was about 12% higher than methods using only an instantaneous spectrum. The results are also comparable with HMM-based methods, although those methods need to treat the time variance of an input vector series with more complicated computations.

Minimum Classification Error 방법 도입을 통한 Gaussian Mixture Model 환경음 인식성능 향상

본 연구에서는 환경음 인식 성능의 향상을 위하여 GMM의 훈련 방식에 MCE 도입을 제안하였다. 이는 환경음 데이터 모델링에 사용할 분류오류함수를 정의할 때 해당 클래스의 로그우도 뿐 아니라 다른 클래스의 로그우도도 같이 고려함으로써 변별력 있는 분류가 이뤄질 수 있게 한다. 모델의 파라미터는 전체 클래스를 고려한 손실함수를 정의하고, GPD(generalized probabilistic descent)알고리즘을 이용하여 추정하였다. 제안된 방법의 인식 성능 비교를 위해 모두 9가지 환경음을 전처리 과정과 MFCC(mel-frequency cepstral coefficients)를 이용하여 12차 특징을 추출하고, 이를 혼합 성분의 수에 따라 GMM 분류 실험을 행하였다. 실험 결과에 따르면 혼합 성분을 19개 사용한 경우에서 MCE 훈련 방식이 평균 87.06%의 인식률로 가장 좋은 성능을 보였다. 이 결과로 제안한 MCE 훈련 방식이 환경음 인식에서 GMM의 훈련 방식으로 효과적으로 사용될 수 있음을 확인하였다. In this paper, we proposed the MCE as a GMM training method to improve the performance of environmental sounds recognition. We model the environmental sounds data with newly defined misclassification function using the log likelihood of the corresponding class and the log likelihood of the rest classes for discriminative training. The model parameters are estimated with the loss function using GPD(generalized probabilistic descent). For recognition performance comparison, we extracted the 12 degrees features using preprocessing and MFCC(mel-frequency cepstral coefficients) of the 9 kinds of environmental sounds and carry out GMM classification experiments. According to the experimental results, MCE training method showed the best performance by an average of 87.06% with 19 mixtures. This result confirmed us that MCE training method could be effectively used as a GMM training method in environmental sounds recognition.

Neural basis of music knowledge: evidence from the dementias

The study of patients with semantic dementia has revealed important insights into the cognitive and neural architecture of semantic memory. Patients with semantic dementia are known to have difficulty understanding the meanings of environmental sounds from an early stage but little is known about their knowledge for famous tunes, which might be preserved in some cases. Patients with semantic dementia (n = 13), Alzheimer's disease (n = 14) as well as matched healthy control participants (n = 20) underwent a battery of tests designed to assess knowledge of famous tunes, environmental sounds and famous faces, as well as volumetric magnetic resonance imaging. As a group, patients with semantic dementia were profoundly impaired in the recognition of everyday environmental sounds and famous tunes with consistent performance across testing modalities, which is suggestive of a central semantic deficit. A few notable individuals (n = 3) with semantic dementia demonstrated clear preservation of knowledge of known melodies and famous people. Defects in auditory semantics were mild in patients with Alzheimer's disease. Voxel-based morphometry of magnetic resonance brain images showed that the recognition of famous tunes correlated with the degree of right anterior temporal lobe atrophy, particularly in the temporal pole. This area was segregated from the region found to be involved in the recognition of everyday sounds but overlapped considerably with the area that was correlated with the recognition of famous faces. The three patients with semantic dementia with sparing of musical knowledge had significantly less atrophy of the right temporal pole in comparison to the other patients in the semantic dementia group. These findings highlight the role of the right temporal pole in the processing of known tunes and faces. Overlap in this region might reflect that having a unique identity is a quality that is common to both melodies and people.

Musical anhedonia: Selective loss of emotional experience in listening to music

Recent case studies have suggested that emotion perception and emotional experience of music have independent cognitive processing. We report a patient who showed selective impairment of emotional experience only in listening to music, that is musical anhednia. A 71-year-old right-handed man developed an infarction in the right parietal lobe. He found himself unable to experience emotion in listening to music, even to which he had listened pleasantly before the illness. In neuropsychological assessments, his intellectual, memory, and constructional abilities were normal. Speech audiometry and recognition of environmental sounds were within normal limits. Neuromusicological assessments revealed no abnormality in the perception of elementary components of music, expression and emotion perception of music. Brain MRI identified the infarct lesion in the right inferior parietal lobule. These findings suggest that emotional experience of music could be selectively impaired without any disturbance of other musical, neuropsychological abilities. The right parietal lobe might participate in emotional experience in listening to music.

Comparison of word and environmental sound recognition by cochlear implant and hearing aid users

인공와우(CI, Cochlear Implant)는 보청기(HA, Hearing Aid)로 재활이 어려운 양측 고심도 감각신경성 난청인들에게 청각정보를 제공해줄 수 있는 방법으로, 송화기에 유입된 음원을 전기에너지로 변환하여 와우 내에 삽입된 전극 통하여 전기자극을 주어 소리 정보를 제공한다. 2005년부터 실시된 한국 정부의 보험급여 확대로 와우 이식의 경제적 부담을 덜게 되어 보다 많은 고심도 난청인이 혜택을 받을 수 있게 되었다. 최근 CI 대상가능 후보가 점차 확대되어 다양한 연령, 다양한 원인에 의한 난청으로 적용 범위가 넓어졌다.

Effects of aging, spectral filtering, and temporal compression on word and environmental sound recognition

어음과 환경음은 인간의 의사소통 시 다양한 정보를 제공하는 중요한 소리이다. 그러나 실제생활에서 다양한 배경소음, 청자의 청력상태, 화자의 발화속도 등에 의해 소리 내 중요한 주파수 혹은 시간적 정보(spectral or temporal information)가 제한될 수 있다. 국내외 다양한 선행 연구는 주파수, 시간적 정보가 감소하면 청자의 소리인지능력이 유의미하게 저하하고 노인일수록 그 저하 정도가 컸음을 밝혔다.

Audio Environment Recognition using Zero Crossing Features and MPEG-7 Descriptors

Problem statement: This study investigated zero crossing features and selected MPEG-7 audio descriptors for environment sound recognition applications such as audio forensics. Approach: The study implemented several experiments focusing on the problems of environment recognition from audio particularly for forensic applications. Results: It was investigated the effect of the temporal zero crossing feature as well as selected MPEG-7 audio low level descriptors on environment sound recognition. The performance was evaluated against a varying number of training sounds and samples per training file. Conclusion/Recommendations: Experimental results showed that higher recognition accuracy is achieved by increasing the number of training files and by decreasing the number of samples per training file. This study presented an audio environment recognition using zero crossing features and MPEG-7 Descriptors.

Multisensory perception of action in posterior temporal and parietal cortices

Environmental events produce many sensory cues for identifying the action that evoked the event, the agent that performed the action, and the object targeted by the action. The cues for identifying environmental events are usually distributed across multiple sensory systems. Thus, to understand how environmental events are recognized requires an understanding of the fundamental cognitive and neural processes involved in multisensory object and action recognition. Here, we investigated the neural substrates involved in auditory and visual recognition of object-directed actions. Consistent with previous work on visual recognition of isolated objects, visual recognition of actions, and recognition of environmental sounds, we found evidence for multisensory audiovisual event-selective activation bilaterally at the junction of the posterior middle temporal gyrus and the lateral occipital cortex, the left superior temporal sulcus, and bilaterally in the intraparietal sulcus. The results suggest that recognition of events through convergence of visual and auditory cues is accomplished through a network of brain regions that was previously implicated only in visual recognition of action.