Acoustic Input Research Articles

Guidelines for Best Practice in the Audiological Management of Adults Using Bimodal Hearing Configurations.

Clinics are treating a growing number of patients with greater amounts of residual hearing. These patients often benefit from a bimodal hearing configuration in which acoustic input from a hearing aid on 1 ear is combined with electrical stimulation from a cochlear implant on the other ear. The current guidelines aim to review the literature and provide best practice recommendations for the evaluation and treatment of individuals with bilateral sensorineural hearing loss who may benefit from bimodal hearing configurations. Specifically, the guidelines review: benefits of bimodal listening, preoperative and postoperative cochlear implant evaluation and programming, bimodal hearing aid fitting, contralateral routing of signal considerations, bimodal treatment for tinnitus, and aural rehabilitation recommendations.

Calculating the acoustic input impedance of a simplified brass instrument as an educational laboratory activity

The concept of acoustic impedance is often difficult for students in introductory acoustics courses to make sense of, especially students without advanced mathematics backgrounds. This work summarizes a laboratory activity for students in a general education musical acoustics class where a simplified brass musical instrument is examined, focusing on how the geometry of the air column affects the input impedance of the instrument. Students are guided through making bore profile measurements for use in a computation of the input impedance. Options for making experimental measurements of the simplified instrument are explained. The laboratory activity was successfully used with students who reported their increased understanding of the acoustics of brass musical instruments.

Cross-modal perception of identity by sound and taste in bottlenose dolphins.

While studies have demonstrated concept formation in animals, only humans are known to label concepts to use them in mental simulations or predictions. To investigate whether other animals use labels comparably, we studied cross-modal, individual recognition in bottlenose dolphins (Tursiops truncatus) that use signature whistles as labels for conspecifics in their own communication. First, we tested whether dolphins could use gustatory stimuli and found that they could distinguish between water and urine samples, as well as between urine from familiar and unfamiliar individuals. Then, we paired playbacks of signature whistles of known animals with urine samples from either the same dolphin or a different, familiar animal. Dolphins investigated the presentation area longer when the acoustic and gustatory sample matched than when they mismatched. This demonstrates that dolphins recognize other individuals by gustation alone and can integrate information from acoustic and taste inputs indicating a modality independent, labeled concept for known conspecifics.

"Where are the . . . Fixations?": Grammatical number cues guide anticipatory fixations to upcoming referents and reduce lexical competition.

Listeners make use of contextual cues during continuous speech processing that help overcome the limitations of the acoustic input. These semantic, grammatical, and pragmatic cues facilitate prediction of upcoming words and/or reduce the lexical search space by inhibiting activation of contextually inappropriate words that share phonological information with the target. The current study used the visual world paradigm to assess whether and how listeners use contextual cues about grammatical number during sentence processing by presenting target words in carrier phrases that were grammatically unconstraining ("Click on the . . .") or grammatically constraining ("Where is/are the . . ."). Prior to the onset of the target word, listeners were already more likely to fixate on plural objects in the "Where are the . . ." context than the "Where is the . . ." context, indicating that they used the construction of the verb to anticipate the referent. Further, participants showed less interference from cohort competitors when the sentence frame made them contextually inappropriate, but still fixated on those words more than on phonologically unrelated distractor words. These results suggest that listeners rapidly and flexibly make use of contextual cues about grammatical number while maintaining sensitivity to the bottom-up input. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Modeling the Repetition-Based Recovering of Acoustic and Visual Sources With Dendritic Neurons.

In natural auditory environments, acoustic signals originate from the temporal superimposition of different sound sources. The problem of inferring individual sources from ambiguous mixtures of sounds is known as blind source decomposition. Experiments on humans have demonstrated that the auditory system can identify sound sources as repeating patterns embedded in the acoustic input. Source repetition produces temporal regularities that can be detected and used for segregation. Specifically, listeners can identify sounds occurring more than once across different mixtures, but not sounds heard only in a single mixture. However, whether such a behavior can be computationally modeled has not yet been explored. Here, we propose a biologically inspired computational model to perform blind source separation on sequences of mixtures of acoustic stimuli. Our method relies on a somatodendritic neuron model trained with a Hebbian-like learning rule which was originally conceived to detect spatio-temporal patterns recurring in synaptic inputs. We show that the segregation capabilities of our model are reminiscent of the features of human performance in a variety of experimental settings involving synthesized sounds with naturalistic properties. Furthermore, we extend the study to investigate the properties of segregation on task settings not yet explored with human subjects, namely natural sounds and images. Overall, our work suggests that somatodendritic neuron models offer a promising neuro-inspired learning strategy to account for the characteristics of the brain segregation capabilities as well as to make predictions on yet untested experimental settings.

PLDA inspired Siamese networks for speaker verification

The deep learning methodologies in state-of-the-art speaker recognition systems are predominantly limited to the extraction of recording level embeddings. This is usually followed by generative modeling of the embeddings to output the verification score. In this paper, we explore a fully neural approach where the neural model outputs the verification score directly, given the acoustic feature inputs. This model, termed as Siamese neural network (SiamNN), combines the embedding extraction and back-end modeling into a single processing pipeline. The back-end modeling is achieved using a neural approach to PLDA modeling, called neural probabilistic linear discriminant analysis (NPLDA). In the NPLDA model, the verification score is computed as a discriminative similarity function. The development of the single neural SiamNN model allows the joint optimization of all the modules using a verification cost. Several speaker recognition experiments are performed using SITW, VOiCES, and NIST SRE datasets where the proposed SiamNN model is shown to significantly improve over the state-of-art x-vector PLDA baseline system (relative improvements of up to 35% in the primary cost metric). We also provide a detailed analysis of the influence of hyper-parameters, choice of loss functions, and data sampling strategies for training the model. In particular, we highlight that the proposed soft detection cost function based optimization improves over other loss functions considered.

Optimal control of the nonspherical oscillations of encapsulated microbubbles

Encapsulated microbubbles (EMBs) were originally developed as contrast agents for ultrasound imaging but are more recently emerging as vehicles for intravenous drug and gene delivery. Ultrasound can excite nonspherical oscillations, or shape modes, that can enhance the acoustic signature of an EMB and also incite rupture, which promotes drug and gene delivery at targeted sites. Therefore, the ability to control shape modes can improve the efficacy of both the diagnosis and treatment mediated by EMBs. This work uses optimal control theory to determine the ultrasound input that maximizes a desired nonspherical EMB response (e.g., to enhance scattering or rupture), while minimizing the total acoustic input in order to enhance patient safety and reduce unwanted side effects. The optimal control problem is applied to a model of an EMB that accounts for small amplitude shape deformations. This model is solved subject to a cost function that maximizes the incidence of rupture or acoustic echo while minimizing the acoustic energy input. The optimal control problem is solved numerically through pseudospectral collocation methods using commercial optimization software. Single frequency and broadband acoustic forcing schemes are explored and compared. The results show that broadband forcing significantly reduces the acoustic effort required to incite EMB rupture relative to single frequency schemes. Furthermore, the acoustic effort required depends strongly on the shape mode that is forced to become unstable.

Frequency-selective acoustic and haptic smart skin for dual-mode dynamic/static human-machine interface.

Accurate transmission of biosignals without interference of surrounding noises is a key factor for the realization of human-machine interfaces (HMIs). We propose frequency-selective acoustic and haptic sensors for dual-mode HMIs based on triboelectric sensors with hierarchical macrodome/micropore/nanoparticle structure of ferroelectric composites. Our sensor shows a high sensitivity and linearity under a wide range of dynamic pressures and resonance frequency, which enables high acoustic frequency selectivity in a wide frequency range (145 to 9000 Hz), thus rendering noise-independent voice recognition possible. Our frequency-selective multichannel acoustic sensor array combined with an artificial neural network demonstrates over 95% accurate voice recognition for different frequency noises ranging from 100 to 8000 Hz. We demonstrate that our dual-mode sensor with linear response and frequency selectivity over a wide range of dynamic pressures facilitates the differentiation of surface texture and control of an avatar robot using both acoustic and mechanical inputs without interference from surrounding noise.

Method to account for natural fracture induced elastic anisotropy in geomechanical characterization of shale gas reservoirs

Shale has been usually recognized as a transverse isotropic (TI) medium in conventional geomechanical log interpretation due to its laminated nature. However, when natural fractures exist in the shale rock, additional elastic anisotropy is introduced, converting laminated Shale to an orthorhombic (OB) medium. Previous studies illustrate that neglecting the natural fracture induced anisotropy in shale geomechanical log interpretation could lead to inaccurate evaluations of elastic moduli and in-situ stresses.In this paper, a new method is developed to account for the natural fracture induced anisotropy in geomechanical log interpretation based upon the TI acoustic model developed by the author and a characterization technique of elastic wave anisotropy (Sayers, 1991). The new OB model incorporates the four acoustic log data inputs and five modeling constraints in a nonlinear optimization algorithm to solve for the nine independent stiffness coefficients of an OB rock, and further to solve for the geomechanical properties and in-situ stress profiles in an OB formation.The new method was validated with a Marcellus Gas Shale field case. Both the new OB model and the conventional TI model were applied to interpret the minimum horizontal stress profile for the same formation. By comparing the results, the OB model is more robust than the TI from two aspects. First, the average stress magnitude predicted by the OB model is closer to the one measured by the Diagnostic Fracture Injection Test (DFIT). Second, the OB model predicts a more obvious stress barrier between the lower Marcellus and upper Onondaga Limestone than the TI model does. The predicted stress barrier is consistent with the observation of the microseismic events of a horizontal well drilled and completed nearby, which reveals that no hydraulic fracture propagates downward through the bottom boundary of Marcellus Shale into the underlying Onondaga Limestone.

Orthography does not hinder non-native production learning in children

Children are known to be fast learners due to their neural plasticity. Learning a non-native language (L2) requires the mastering of new production patterns. In classroom settings, learners are not only exposed to the acoustic input, but also to the unfamiliar grapheme–phoneme correspondences of the L2 orthography. We tested how 9–10-year-old children, with Finnish as a native language (L1), respond to a two-day listen-and-repeat training paradigm, where they simultaneously hear acoustic stimuli and see orthographic cues. In the procedure, non-words containing the L2 vowel /ʉ/ were presented simultaneously with an orthographic cue showing <u>, guiding pronunciation towards the L1 vowel /u/ according to Finnish grapheme–phoneme correspondences. Earlier studies showed that Finnish adults rely on the orthographic cue over the acoustic one, leading them to produce /u/ instead of /ʉ/ when presented with the incongruent L1–L2 grapheme–phoneme correspondence (<u> – L1: /u/, L2: /ʉ/). Also, an earlier result from age-matched children receiving only acoustic input showed relatively fast pronunciation changes towards the target vowel. Our present results indicate clear and fast production learning of the non-native sound, and the misleading orthographic cue did not draw attention away from the target acoustic form. With orthographic cues, the participants learned to produce novel sounds faster than without them.

Person Localization Model Based on a Fusion of Acoustic and Visual Inputs

PLEA is an interactive, biomimetic robotic head with non-verbal communication capabilities. PLEA reasoning is based on a multimodal approach combining video and audio inputs to determine the current emotional state of a person. PLEA expresses emotions using facial expressions generated in real-time, which are projected onto a 3D face surface. In this paper, a more sophisticated computation mechanism is developed and evaluated. The model for audio-visual person separation can locate a talking person in a crowded place by combining input from the ResNet network with input from a hand-crafted algorithm. The first input is used to find human faces in the room, and the second input is used to determine the direction of the sound and to focus attention on a single person. After an information fusion procedure is performed, the face of the person speaking is matched with the corresponding sound direction. As a result of this procedure, the robot could start an interaction with the person based on non-verbal signals. The model was tested and evaluated under laboratory conditions by interaction with users. The results suggest that the methodology can be used efficiently to focus a robot’s attention on a localized person.

The XMUSPEECH System for Accented English Automatic Speech Recognition

In this paper, we present the XMUSPEECH systems for Track 2 of the Interspeech 2020 Accented English Speech Recognition Challenge (AESRC2020). Track 2 is an Automatic Speech Recognition (ASR) task where the non-native English speakers have various accents, which reduces the accuracy of the ASR system. To solve this problem, we experimented with acoustic models and input features. Furthermore, we trained a TDNN-LSTM language model for lattice rescoring to obtain better results. Compared with our baseline system, we achieved relative word error rate (WER) improvements of 40.7% and 35.7% on the development set and evaluation set, respectively.

Parallel processing in speech perception with local and global representations of linguistic context.

Speech processing is highly incremental. It is widely accepted that human listeners continuously use the linguistic context to anticipate upcoming concepts, words, and phonemes. However, previous evidence supports two seemingly contradictory models of how a predictive context is integrated with the bottom-up sensory input: Classic psycholinguistic paradigms suggest a two-stage process, in which acoustic input initially leads to local, context-independent representations, which are then quickly integrated with contextual constraints. This contrasts with the view that the brain constructs a single coherent, unified interpretation of the input, which fully integrates available information across representational hierarchies, and thus uses contextual constraints to modulate even the earliest sensory representations. To distinguish these hypotheses, we tested magnetoencephalography responses to continuous narrative speech for signatures of local and unified predictive models. Results provide evidence that listeners employ both types of models in parallel. Two local context models uniquely predict some part of early neural responses, one based on sublexical phoneme sequences, and one based on the phonemes in the current word alone; at the same time, even early responses to phonemes also reflect a unified model that incorporates sentence-level constraints to predict upcoming phonemes. Neural source localization places the anatomical origins of the different predictive models in nonidentical parts of the superior temporal lobes bilaterally, with the right hemisphere showing a relative preference for more local models. These results suggest that speech processing recruits both local and unified predictive models in parallel, reconciling previous disparate findings. Parallel models might make the perceptual system more robust, facilitate processing of unexpected inputs, and serve a function in language acquisition.

Correlates of deviance detection in auditory brainstem responses of bats

Identifying unexpected acoustic inputs, which allows to react appropriately to new situations, is of major importance for animals. Neural deviance detection describes a change of neural response strength to a stimulus solely caused by the stimulus' probability of occurrence. In the present study, we searched for correlates of deviance detection in auditory brainstem responses obtained in anaesthetised bats (Carollia perspicillata). In an oddball paradigm, we used two pure tone stimuli that represented the main frequencies used by the animal during echolocation (60 kHz) and communication (20 kHz). For both stimuli, we could demonstrate significant differences of response strength between deviant and standard response in slow and fast components of the auditory brainstem response. The data suggest the presence of correlates of deviance detection in brain stations below the inferior colliculus (IC), at the level of the cochlea nucleus and lateral lemniscus. Additionally, our results suggest that deviance detection is mainly driven by repetition suppression in the echolocation frequency band, while in the communication band, a deviant-related enhancement of the response plays a more important role. This finding suggests a contextual dependence of the mechanisms underlying subcortical deviance detection. The present study demonstrates the value of auditory brainstem responses for studying deviance detection and suggests that auditory specialists, such as bats, use different frequency-specific strategies to ensure an appropriate sensation of unexpected sounds.

Pitch Perception With the Temporal Limits Encoder for Cochlear Implants.

The temporal-limits-encoder (TLE) strategy has been proposed to enhance the representation of temporal fine structure (TFS) in cochlear implants (CIs), which is vital for many aspects of sound perception but is typically discarded by most modern CI strategies. TLE works by computing an envelope modulator that is within the temporal pitch limits of CI electric hearing. This paper examines the TFS information encoded by TLE and evaluates the salience and usefulness of this information in CI users. Two experiments were conducted to compare pitch perception performance of TLE versus the widely-used Advanced Combinational Encoder (ACE) strategy. Experiment 1 investigated whether TLE processing improved pitch discrimination compared to ACE. Experiment 2 parametrically examined the effect of changing the lower frequency limit of the TLE modulator on pitch ranking. In both experiments, F0 difference limens were measured with synthetic harmonic complex tones using an adaptive procedure. Signal analysis of the outputs of TLE and ACE strategies showed that TLE introduces important temporal pitch cues that are not available with ACE. Results showed an improvement in pitch discrimination with TLE when the acoustic input had a lower F0 frequency. No significant effect of lower frequency limit was observed for pitch ranking, though a lower limit did tend to provide better outcomes. These results suggest that the envelope modulation introduced by TLE can improve pitch perception for CI listeners.

NAAQA: A Neural Architecture for Acoustic Question Answering.

The goal of the Acoustic Question Answering (AQA) task is to answer a free-form text question about the content of an acoustic scene. It was inspired by the Visual Question Answering (VQA) task. In this paper, based on the previously introduced CLEAR dataset, we propose a new benchmark for AQA, namely CLEAR2, that emphasizes the specific challenges of acoustic inputs. These include handling of variable duration scenes, and scenes built with elementary sounds that differ between training and test set. We also introduce NAAQA, a neural architecture that leverages specific properties of acoustic inputs. The use of 1D convolutions in time and frequency to process 2D spectro-temporal representations of acoustic content shows promising results and enables reductions in model complexity. We show that time coordinate maps augment temporal localization capabilities which enhance performance of the network by ∼17 percentage points. On the other hand, frequency coordinate maps have little influence on this task. NAAQA achieves 79.5% of accuracy on the AQA task with ∼four times fewer parameters than the previously explored VQA model. We evaluate the performance of NAAQA on an independent data set reconstructed from DAQA. We also test the addition of a MALiMo module in our model on both CLEAR2 and DAQA. We provide a detailed analysis of the results for the different question types. We release the code to produce CLEAR2 as well as NAAQA to foster research in this newly emerging machine learning task.

Parametric model of young infants’ eardrum and ear canal impedances supporting immittance measurement results. Part I: Development of the model

Wideband acoustic immitance (WAI) measurements provide an objective means to detect pathological middle ear conditions. However, for ears of young infants, it is still difficult to make clear statements about the middle ear status based on WAI measurements. In order to gain a better understanding of WAI data obtained in young infants’ ears, a parametric electro-acoustic model of the ear canal and the middle ear of young infants is proposed. In this first part of the two-part paper, the development of the model for the healthy ear is presented. Based on an existing model for adult ears, the presented model is adapted to young infants’ ears, uses parameters suited to represent physiological properties, and uses a smaller number of parameters in order to reduce model complexity. A comparison of the acoustic input impedance of the ear predicted by the model with real ear measurements in young infants’ ears showed a good agreement in the main characteristics. Model predictions show that the medium frequency range (about 1–3 kHz) of the acoustic input impedance of the ear is dominated by the properties of the eardrum and the middle ear, indicating that pathological middle ear conditions can preferably be detected in this frequency range.

Implications of wind and vessel noise on the sound fields experienced by southern resident killer whales Orcinus orca in the Salish Sea

The soundscape of critical habitat for southern resident killer whale (SRKW) Orcinus orca in the Salish Sea, the waters around southern British Columbia, Canada, and northern Washington State, USA, is shaped by wind and wave noise as well as heavy commercial and recreational vessel traffic loads. First, we used recordings from 6 passive acoustic moorings to characterize the acoustic landscape experienced by SRKW in this region, focusing on the frequencies used for communication and echolocation. Mid-frequency wind noise was prevalent in winter sound fields, whereas higher-frequency noise levels associated with increased numbers of recreational vessels increased during summer. Commercial vessel presence was consistent, with acoustic inputs prevalent in the western part of the study area. The potential implications of these additions on SRKW acoustics use were then explored for the frequency band 1-40 kHz to represent communication calls and at 50 kHz to consider echolocation. The inputs of wind were extrapolated from modelled hourly wind speed measures and commercial shipping noise. The noise impact was expressed as a proportional reduction of communication and echolocation extent compared to maximum acoustic ranges at ‘minimum ambient’ levels, void of vessel and abiotic noise. The reductions calculated were substantial, with the presence and impact of vessel noise greater than wind-derived additions and the greatest impacts around shipping lanes. Impacts were found for SRKW foraging areas, with implications for group cohesion and feeding success. This interpretation of the influence of natural and vessel noise more clearly demonstrates the potential implications of altered soundscapes for SRKW.

Investigation on Acoustic Versus Functional Characteristics of Porous Asphalt

The theoretical background, standards, and contract requirements of pavement friction courses involve functional (e.g., permeability) and acoustic (e.g., resistivity) characteristics. Unfortunately, their relationship is partly unknown and uncertain. This affects the comprehensiveness and soundness of the mix design of asphalt pavements. Based on the issues above, the goals of this study were confined into the following ones: 1) to investigate the relationship between acoustic and functional properties of porous asphalts; 2) to investigate, through one-layer (1L) and two-layer (2L) models, the effectiveness of the estimates of acoustic input data through mixture volumetric- and permeability-related characteristics. Volumetric and acoustic tests were performed and simulations were carried out. Equations and strategies to support a comprehensive approach were derived. Results demonstrate that even if the measured resistivity is very important, permeability-based estimates of resistivity well explain acoustic spectra. Furthermore, the distance between observed and estimated peaks of the absorption spectrum emerges as the best error function.

Speech Recorder and Translator using Google Cloud Speech-to-Text and Translation

The most popular video website YouTube has about 2 billion users worldwide who speak and understand different languages. Subtitles are essential for the users to get the message from the video. However, not all video owners provide subtitles for their videos. It causes the potential audiences to have difficulties in understanding the video content. Thus, this study proposed a speech recorder and translator to solve this problem. The general concept of this study was to combine Automatic Speech Recognition (ASR) and translation technologies to recognize the video content and translate it into other languages. This paper compared and discussed three different ASR technologies. They are Google Cloud Speech-to-Text, Limecraft Transcriber, and VoxSigma. Finally, the proposed system used Google Cloud Speech-to-Text because it supports more languages than Limecraft Transcriber and VoxSigma. Besides, it was more flexible to use with Google Cloud Translation. This paper also consisted of a questionnaire about the crucial features of the speech recorder and translator. There was a total of 19 university students participated in the questionnaire. Most of the respondents stated that high translation accuracy is vital for the proposed system. This paper also discussed a related work of speech recorder and translator. It was a study that compared speech recognition between ordinary voice and speech impaired voice. It used a mobile application to record acoustic voice input. Compared to the existing mobile App, this project proposed a web application. It was a different and new study, especially in terms of development and user experience. Finally, this project developed the proposed system successfully. The results showed that Google Cloud Speech-to-Text and Translation were reliable to use in video translation. However, it could not recognize the speech when the background music was too loud. Besides, it had a problem of direct translation, which was challenging. Thus, future research may need a custom trained model. In conclusion, the proposed system in this project was to contribute a new idea of a web application to solve the language barrier on the video watching platform.